This article organises potential areas of criticism or challenges embedded in the design and administration of standardised assessments of learning levels in order to promote dialogue and research on educational assessments. The article begins by addressing debates around epistemological claims: issues that pertain to testing in general and issues that are particular to standardised testing. Then, it addresses some political attributes of international tests so as to situate the debates beyond feasibility, attributes and scope-related issues. The article claims that the field of education testing has identified a number of issues and challenges stemming from diversity, and has developed methods and procedures to address many of them. From this viewpoint, testing is just like any other domain of scientific enquiry. However, international assessments of learning outcomes are not necessarily, or primarily, scientific endeavours; they are political devices and therefore should be scrutinised considering scientific attributes as well as some political features that, even if intertwined with technicalities, go well beyond them. Thus, critiques of international assessments would be better framed if their political attributes are taken as organising principles of the criticism, alongside those elements that pertain to their technical attributes, since these are not incidental but deeply interlinked.