A major challenge remains to understand the relative contributions of history, dispersal, and environmental filtering to the assembly of hyperdiverse communities across spatial scales. Here, we examine the extent to which biogeographical history and habitat specialization have generated turnover among and within lineages of Amazonian trees across broad geographic and environmental gradients. We replicated standardized tree inventories in 102 0.1-ha plots located in two distant regions—the western Amazon and the eastern Guiana shield. Within each region, we used a nested design to replicate plots on contrasted habitats: white-sand, terra firme, and seasonally flooded forests. Our plot network encompassed 26,386 trees that together represented 2,745 distinct taxa, which we standardized across all plots and regions. We combined taxonomic and phylogenetic data with detailed soil measurements and climatic data to: (1) test whether patterns of taxonomic and phylogenetic composition are consistent with recent or historical processes, (2) disentangle the relative effects of habitat, environment, and geographic distance on taxonomic and phylogenetic turnover among plots, and (3) contrast the proportion of habitat specialists among species from each region. We found substantial species turnover between Peru and French Guiana, with only 8.8% of species shared across regions; genus composition remained differentiated across habitats and regions, whereas turnover at higher taxonomic levels (family, order) was much lower. Species turnover across plots was explained primarily by regions, but also substantially by habitat differences and to a lesser extent by spatial distance within regions. Conversely, the composition of higher taxonomic levels was better explained by habitats (especially comparing white-sand forests to other habitats) than spatial distance. White-sand forests harbored most of the habitat specialists in both regions, with stronger habitat specialization in Peru than in French Guiana. Our results suggest that recent diversification events have resulted in extremely high turnover in species and genus composition with relatively little change in the composition of higher lineages. Our results also emphasize the contributions of rare habitats, such as white-sand forests, to the extraordinary diversity of the Amazon and underline their importance as conservation priorities.