CHALLENGES OF SAMPLING AND REPRESENTATION IN PISA STUDIES: IMPLICATIONS FOR STEM READINESS RESEARCH

Abstract

The Programme for International Student Assessment (PISA) has become one of the most influential international large-scale assessments, offering valuable insights into global education systems, particularly in evaluating students’ readiness for STEM (Science, Technology, Engineering, and Mathematics). However, the reliability of PISA outcomes often depends on the robustness of sampling and representation strategies employed in various countries. This study critically examines the methodological challenges associated with PISA’s sampling frameworks and their implications for interpreting STEM readiness trends. Using a mixed-method approach, the paper reviews PISA technical documentation and applies secondary statistical analysis to cross-national datasets from PISA 2018, focusing on science and mathematics literacy scores. Descriptive and inferential statistics were employed to highlight disparities arising from non-response bias, underrepresentation of rural and marginalized populations, and oversampling of high-performing schools. The analysis reveals that while PISA adheres to rigorous OECD sampling standards, practical implementation varies significantly across countries, leading to distortions in comparability and potential misrepresentation of STEM readiness levels. For example, logistic regression models demonstrate how socio-economic background and school type amplify sampling errors, while chi-square tests confirm significant deviations in participation rates among certain demographic groups. The findings underscore three central challenges: (i) ensuring inclusivity of diverse student populations; (ii) mitigating attrition and non-response bias; and (iii) balancing national sampling autonomy with international comparability. These limitations question the validity of using PISA outcomes as straightforward indicators of STEM readiness without contextual adjustments. The study concludes that robust methodological refinements—such as adaptive sampling techniques, transparent data disclosure, and contextual weighting—are essential for strengthening the credibility of PISA-based STEM research. By addressing these challenges, educational policymakers and researchers can better interpret global trends and design equitable interventions to enhance STEM readiness worldwide.