The indicator used to measure progress toward the Millennium Development Goal (MDG) for water is access to an improved water supply. However, improved supplies are frequently fecally contaminated in developing countries. We examined factors associated with Escherichia coli contamination of improved water supplies in rural Pisco province, Peru. A random sample of 207 households with at least one child less than 5 years old was surveyed, and water samples from the source and storage container were tested for E. coli contamination. Although over 90% of households used an improved water source, 47% of source and 43% of stored water samples were contaminated with E. coli. Pouring or using a spigot to obtain water from the storage container instead of dipping a hand or object was associated with decreased risk of contamination of stored water (adjusted prevalence ratio [aPR] = 0.58, 95% confidence interval [CI] = 0.42, 0.80). Container cleanliness (aPR = 0.67, 95% CI = 0.45, 1.00) and correct handwashing technique (aPR = 0.62, 95% CI = 0.42, 0.90) were also associated with decreased contamination risk. These findings highlighted the limitations of improved water supplies as an indicator of safe water access. To ensure water safety in the home, household water treatment and improved hygiene, water handling, and storage practices should be promoted.