Contrary to expectations, some human-modified landscapes are considered to sustain both human activities and biodiversity over the long-term. Agroforestry systems are among these landscapes where crops are planted under native shade trees. In this context, ancient agroforestry systems can provide insight into how farmers managed the landscape over time. Such insight can help to quantify the extent to which tropical forests (especially habitat-specialist trees) are responding to local and landscape-level management. Here, we extracted fossil pollen (indicator of past vegetation changes) and macroscopic charcoal (indicator of biomass burning) from four forest hollows’ sedimentary sequences in an ancient agroforestry system in Western Ghats, India. We used a mixed-modelling approach and a principal components analysis (PCA) to determine past trajectories of forest change and species composition dynamics for the last 900 years. In addition, we reconstructed the long-term forest canopy dynamics and examined the persistence of habitat-specialist trees over time. Our results show that the four sites diverged to a surprising degree in both taxa composition and dynamics. However, despite these differences, forest has persisted over 900 years under agricultural activities within agroforestry systems. This long-term analysis highlights the importance of different land-use legacies as a framework to increase the effectiveness of management across tropical agricultural lands.