A key premise underlying discussion about deforestation in Amazonian Peru is that small-scale or so-called migratory agriculture is the main driver of deforestation. This premise has been expressed in government documents and public outreach events. How the Peruvian government understands drivers of deforestation in the Amazon has profound implications for how it will confront the problem. It is therefore important to critically revisit assumptions underlying this narrative. We find that the narrative is based on remote sensing of deforestation patch sizes but not on field data, potentially conflating distinct drivers of deforestation under the umbrella of “migratory,” “small-scale,” or “subsistence” agriculture. In fact, small patches of deforested land may indicate any number of processes, including sustainable fallow management and agroforestry. Moreover, the data underlying the narrative tell us little about the actors driving these processes or their motivations. Different processes have distinct implications for environmental sustainability and require targeted policy responses. We unpack these diverse actors, geographies, and motivations of small-patch deforestation in the Peruvian Amazon and argue that differentiating among these drivers is necessary to develop appropriate policy responses. We call for researchers to revisit assumptions and critically assess the motivations of observed deforestation to appropriately target policy action.