Background: The study examined the association of body size (weight and length) at birth and gain in height and weight during childhood and adolescence with areal bone mineral density (aBMD) in adulthood for women and men. Methods: 756 members (335 men and 421 women) of the 1993 Pelotas (Brazil) Birth cohort were studied. Data on weight and length/height were obtained at birth and subsequent follow-ups at 1, 4, 11, 15, 18, and 22 years of age and specific z scores were calculated by sex. The outcome was whole body aBMD (g/cm²) measured at 22 years of age using dual-energy X-ray absorptiometry (DXA). The effects of exposures, weight and length/height gain, were analyzed using conditional relative weight (CWh) and conditional length/height (CH). Linear regression models were adjusted for multiple confounders, including mother's educational level, family income, maternal smoking during pregnancy, gestational age, breastfeeding and skin color. Results: In the adjusted models, among men greater height gain at 4, 11, and 18 years of age was associated with higher whole body aBMD, and the result with greatest magnitude was at 11 years of age (β 0.018 g/cm²; 95%CI 0.006; 0.030). Among women, aBMD was associated with height gain at all assessments from 1–15 years, with greatest effect size at 4 years of age (β 0.017 g/cm²; 95%CI 0.007; 0.027). Regarding to body weight, among men, greater weight at 4 and 15 years were associated with higher aBMD, with the highest coefficients for 15 years of age (β 0.015 g/cm²; 95%CI 0.003; 0.027); for women, except at birth, all weight gain variables were associated with aBMD and the highest coefficients were observed at 4 years (β 0.025 g/cm²; 95%CI 0.015; 0.035). Conclusions: In this birth cohort, height and weight gain, especially from 4 to 15 years have important positive implications for aBMD to early adulthood.