Occupational position is one of the determinants of psychological health, but this association may differ for men and women depending on the social context. In contemporary Japanese society, occupational gender segregation persists despite increased numbers of women participating in the labour market, which may contribute to gender specific patterns in the prevalence of poor psychological health. The present study examined gender specific associations between occupational position and psychological health in Japan, and the potential mediating effects of job control and effort-reward imbalance in these associations. We used data obtained from 7123 men and 2222 women, aged between 18 and 65 years, who participated in an occupational cohort study, the Japanese Study of Health, Occupation, and Psychosocial Factors Related Equity (J-HOPE), between 2011 and 2012. We used logistic regression to examine the association between occupational position and poor psychological health, adjusted for age, working hours, household income and education, as well as psychosocial work characteristics (job control and effort-reward imbalance). The prevalence of poor psychological health increased from manual/service occupations (23%) to professionals/managers (38%) among women, while it did not vary by occupational position among men. In women, the significant association between occupational position and psychological health was not explained by job control, but was attenuated by effort-reward imbalance. Our findings suggest that Japanese women in more advantaged occupational positions are likely to be at a greater risk for poor psychological health due to higher levels of effort-reward imbalance at work.