Purpose: To evaluate the role of urbanization as an effect modifier for the association between specific chronic conditions and number of conditions with health-related quality of life (QOL). Methods: We analyzed cross-sectional data from the CRONICAS Cohort Study conducted in Lima (highly urbanized), Tumbes (semi-urban), as well as rural and urban sites in Puno. Exposures of interest were chronic bronchitis, depressive mood, hypertension, type 2 diabetes, and a composite variable aggregating the number of chronic conditions (the four exposures plus heart disease and stroke). QOL outcomes were assessed with EuroQol’s EQ-5D visual analogue scale (EQ-VAS). We fitted linear regressions with robust variance to evaluate the associations of interest. Study site was assessed as a potential effect modifier using the likelihood-ratio (LR) test. Results: We evaluated data on 2433 subjects: 51.3% were female, mean age was 57.2 years. Study site was found to be an effect modifier only for the association between depressive mood and EQ-VAS score (LR test p < 0.001). Compared to those without depressive mood, participants with depressive mood scored −13.7 points on the EQ-VAS in Lima, −7.9 in urban Puno, −11.0 in semi-urban Tumbes, and −2.7 in rural Puno. Study site was not found to be an effect modifier for the association between the number of chronic conditions and EQ-VAS (LR test p = 0.64). Conclusion: The impact of depressive mood on EQ-VAS was larger in urban than in rural sites, while site was not an effect modifier for the remaining associations.