The present study aimed to clarify the associations among radiation exposure or psychological exposure to the Fukushima nuclear power plant accident (i.e., fear/anxiety immediately after the accident), current radiation anxiety, and psychological distress among non-evacuee community residents in Fukushima five years after the Great East Japan Earthquake, which occurred in March 2011. A questionnaire survey was administered to a random sample of non-evacuee community residents from 49 municipalities of Fukushima prefecture from February to April 2016, and data from 1684 respondents (34.4%) were analyzed. Environmental radiation levels at the time of the accident were ascertained from survey meter data, while environmental radiation levels at the time of the survey were ascertained from monitoring post data. In the questionnaire, immediate fear/anxiety after the accident, current radiation anxiety, and psychological distress were measured using a single-item question, a 7-item scale, and K6, respectively. Multilevel linear or logistic regression models were applied to analyze the determinants of radiation anxiety and psychological distress. The findings showed that environmental radiation levels at the time of the survey were more strongly associated with radiation anxiety than radiation levels immediately after the accident. Disaster-related experiences, such as direct damage, disaster-related family stress, and fear/anxiety after the accident, and demographic characteristics (e.g., younger age, being married, low socioeconomic status) were significantly associated with radiation anxiety. Environmental radiation levels at the time of the accident or survey were not significantly associated with psychological distress. Radiation anxiety largely mediated the association between fear/anxiety after the accident and psychological distress. In addition to environmental radiation levels, respondents’ radiation anxiety was affected by multiple factors, such as disaster-related experiences and demographic characteristics. Radiation levels were not associated with psychological distress in non-evacuee community residents. Rather, fear/anxiety after the nuclear power plant accident may be a determinant of psychological distress, mediated by radiation anxiety.