Remix.run Logo
rawgabbit a day ago

I found this nugget on page 315. It seems to say the OECD acknowledges students are just blowing off this test.

     For most students, PISA is a low-stakes assessment: they can refuse to participate in the test with no negative consequences and do not receive any feedback on their performance. There is a risk, therefore, that students do not do their best on the test; and that they behave differently from what they would do in other circumstances.
    ...In most countries/economies, students reported making less effort on the test in 2025 than in 2022 and in 2018: on average across 35 OECD countries, the difference corresponds to -0.5 points on the 10-point scale (compared to 2018; and -0.3 points compared to 2022) (Table I.A1.1). Meanwhile, the proportion of students who indicated that they invested very little effort in the PISA test increased from 3.5% to 5.4%.

There is also this strange entry here(https://www.oecd.org/en/publications/pisa-2025-results-volum...).

     In this section, the analysis focuses on two types of incorrect responses that can be identified on most, if not all, items: skipped responses (item responses left blank, in-between non-missing responses); and hasty responses (item responses that were not scored as correct, and on which students spent less time than the fastest, yet accurate respondents). More precisely, a “minimum threshold” of time effort required for each item was defined as the 1st percentile of the (unweighted) distribution of response times among all respondents who solved the item correctly, across all participating countries and economies.2 This means that only 1% of all full-credit respondents were able to respond to the item in less time, and 99% of full-credit respondents spent longer than this threshold. The minimum threshold of time required was increased to 10 seconds in all cases where, mostly because of correct guess answers, more than 1% of full-credit respondents had provided the correct response in less than 10 seconds. Respondents who did not obtain a full-credit score on the item, but who spent less time than this threshold, are considered to have given a hasty, speeded response (as defined, a “skipped” response may also be a “hasty” response).
     Table I.A1.12 shows how the proportion of skipped and speeded responses evolved between 2018 and 2025. A clear trend emerges: on average across OECD countries, the average proportion of hasty (incorrect and fast) responses increased significantly between 2018 and 2025. The average proportion of hasty responses increased by over 4 percentage points over the period, reaching a total of about 9%. The average increase across items of hasty responses exceeded 10 percentage points in three countries/economies (Kosovo, Dominican Republic and Bulgaria, in descending order of that increase). Similar analyses for the mathematics and science test are presented in Tables I.A1.13 and I.A1.11; analyses for math, in particular, also show an increase in the proportion of hasty responses between 2022 and 2025, with the largest increases observed in Serbia, Malta and Israel.
    In the PISA test, sequences of items that are left blank at the end of the test are not considered when scoring students’ answers. Similarly, in these analyses, the corresponding students and (missing) answers are excluded from the percentages of “skipped” or “hasty” respondents. Indeed, in PISA, these items are considered as if they were not administered to students, who, most likely, did not reach them. Such non-reached items are also ignored when estimating item difficulty and student performance. However, because, over time, students became more likely to skip or to provide a fast (if incorrect) response rather than to continue working on a task that they perceived as difficult, it is interesting to note that the proportion of non-reached items declined in reading, by about 2 percentage points (Table I.A1.12), on average across OECD countries.
rayiner 18 hours ago | parent [-]

This methodology makes no sense lol. I thought PISA was the gold standard for international educational comparisons.