Artisanal and small-scale gold mining (ASGM) is a primary contributor to global mercury and its rapid expansion raises concern for human exposure. Non-occupational exposure risks are presumed to be strongly tied to environmental contamination; however, the relationship between environmental and human mercury exposure, how exposure has changed over time, and risk factors beyond fish consumption are not well understood in ASGM settings. In Peruvian riverine communities (n = 12), where ASGM has increased 4–6 fold over the past decade, we provide a large-scale assessment of the connection between environmental and human mercury exposure by comparing total mercury contents in human hair (2-cm segment, n = 231) to locally caught fish tissue, analyzing temporal exposure in women of child bearing age (WCBA, 15–49 years, n = 46) over one year, and evaluating general mercury exposure risks including fish and non-fish dietary items through household surveys and linear mixed models. Calculations of an individual’s oral reference dose using the total mercury content in locally-sourced fish underestimated the observed mercury exposure for individuals in many communities. This discrepancy was particularly evident in communities upstream of ASGM, where mercury levels in river fish, water, and sediment measurements from a previous study were low, yet hair mercury was chronically elevated. Hair from 86% of individuals and 77% of children exceeded a USEPA (U.S. Environmental Protection Agency) provisional level (1.2 μg/g) that could result in child developmental impairment. Chronically elevated mercury exposure was observed in the temporal analysis in WCBA. If the most recent exposure exceeded the USEPA level, there was a 97% probability that the individual exceeded that level 8–10 months of the previous year. Frequent household consumption of some fruits (tomato, banana) and grains (quinoa) was significantly associated with 29–75% reductions in hair mercury. Collectively, these data demonstrate that communities located hundreds of kilometers from ASGM are vulnerable to chronically elevated mercury exposure. Furthermore, unexpected associations with fish mercury contents and non-fish dietary intake highlight the need for more in-depth analyses of exposure regimes to identify the most vulnerable populations and to establish potential interventions.