The PLOS ONE Collection "Measuring forest conservation effectiveness" brings together a series of studies that evaluate the effectiveness of tropical forest conservation policies and programs with the goal of measuring conservation success and associated co-benefits. This overview piece describes the geographic and methodological scope of these studies, as well as the policy instruments covered in the Collection as of June 2016. Focusing on forest cover change, we systematically compare the conservation effects estimated by the studies and discuss them in the light of previous findings in the literature. Nine studies estimated that annual conservation impacts on forest cover were below one percent, with two exceptions in Mexico and Indonesia. Differences in effect sizes are not only driven by the choice of conservation measures. One key lesson from the studies is the need to move beyond the current scientific focus of estimating average effects of undifferentiated conservation programs. The specific elements of the program design and the implementation context are equally important factors for understanding the effectiveness of conservation programs. Particularly critical will be a better understanding of the causal mechanisms through which conservation programs have impacts. To achieve this understanding we need advances in both theory and methods.