Objectives: This study aimed to compare the longitudinal change of the psychological distress of healthcare workers (HCWs) with non-HCWs during the repeated outbreaks of the COVID-19 in Japan. Methods: The data were retrieved from the Employee Cohort Study in the Covid-19 pandemic in Japan study. An online survey was conducted on March 2020 (T1), on May 2020 (T2), on August 2020 (T3), and on November 2020 (T4). Psychological distress was measured by the Brief Job Stress Questionnaire. A mixed-model repeated-measures ANOVA was conducted as an indicator of the group differences. Results: A total sample of analysis was n = 996 (HCWs, n = 111; non-HCWs, n = 885). HCWs consisted of physicians/nurses/midwives and other HCWs (eg, pharmacists, clinical laboratory technicians) in the clinical settings (n = 19; 17% and n = 61; 55%, respectively), and HCWs not working in the clinical settings (n = 31; 28%). Being HCWs were associated with a significant increase in psychological distress from T1 to T2, T3 and T4 (P =.001, P =.002, P <.001; respectively). Conclusions: The mental health of HCWs deteriorated through the COVID-19 outbreaks compared with non-HCWs. HCWs are continuously the important targets to provide mental health support.