Age-related cognitive decline (ACD) is the gradual process of decreasing of cognitive function over age. Most genetic risk factors for ACD have been identified in European populations and there are no reports in admixed Latin American individuals. We performed admixture mapping, genome-wide association analysis (GWAS), and fine-mapping to examine genetic factors associated with 15-year cognitive trajectory in 1,407 Brazilian older adults, comprising 14,956 Mini-Mental State Examination measures. Participants were enrolled as part of the Bambuí-Epigen Cohort Study of Aging. Our admixture mapping analysis identified a genomic region (3p24.2) in which increased Native American ancestry was significantly associated with faster ACD. Fine-mapping of this region identified a single nucleotide polymorphism (SNP) rs142380904 (β = −0.044, SE = 0.01, p = 7.5 × 10−5) associated with ACD. In addition, our GWAS identified 24 associated SNPs, most in genes previously reported to influence cognitive function. The top six associated SNPs accounted for 18.5% of the ACD variance in our data. Furthermore, our longitudinal study replicated previous GWAS hits for cognitive decline and Alzheimer’s disease. Our 15-year longitudinal study identified both ancestry-specific and cosmopolitan genetic variants associated with ACD in Brazilians, highlighting the need for more trans-ancestry genomic studies, especially in underrepresented ethnic groups.