Are PISA Top-Performers also Good Citizens?

Rolf Strietholt, Olesya Gladushyna & Andrés Strello

The education community across the globe is anticipating the release of the latest PISA rankings. These gauge the progress being made by 15-year-olds in reading, science and mathematics across a wide selection of developed and developing countries. One of the widely-acknowledged limitations of PISA as a measure of educational quality is that it covers only a limited range of outputs from an education system. There is hence a threat that, if countries focus efforts upon maximising performance in the PISA tests, then they may divert attention away from important (yet unmeasured) contributions of education to wider society. This includes, for instance, the development of morals and civic engagement. Indeed, in this blog we provide new evidence on the discrepancy between the performance of countries in PISA and children’s knowledge of civics.

 Scant Rankings Provide a Fragmentized Picture

To understand why PISA provides a too simplistic picture of educational quality, it is worth reconsidering the role of education for society. In democratic societies, there is a general consensus that schools must not only prepare children for the workplace, but also to promote social values and active citizenship.

PISA focuses on how education contributes to economic growth and development. For this purpose, the OECD, organization that administers PISA, assesses math, reading, and science literacy which are considered by OECD experts to be the key areas of human capital. Even though knowledge in these academic domains is important for other purposes of education, active citizenship requires further understanding about issues such as human rights, freedom of the press, or financial donations to political parties. In the “post-truth” era when radical populist movements are on the rise worldwide, such civic education is becoming an increasingly important issue. Yet, such knowledge is not well represented by PISA.

Comparing Student Performance in Mathematics and Civic Education

There are, however, alternatives to PISA where the measurement of children’s civic knowledge and attitudes is a central goal. The most widely known cross-national study in this area is the International Civic and Citizenship Education Study (ICCS). Hence, by drawing upon data from both PISA and ICCS, one can see how the academic and civic knowledge of young people across different countries compare. We illustrate such a comparison in Figure 1, which compares average PISA 2015 mathematics scores (horizontal axis) to average scores on the civic knowledge test of ICCS 2016. This comparison has been done for all European and East Asian that participated in both assessments (Latin American countries were excluded because they frequently perform much lower on international assessments).

This figure clearly illustrates that there is huge variation in civic knowledge amongst countries from a similar geographical, historical and cultural space – and who have children with similar PISA test scores. For instance, Hong Kong, Korea, and Chinese Taipei are all PISA top-performers in mathematics, but their results in civic knowledge are up to 66 test points apart (equivalent to more than one additional year of schooling). Similarly, although Danish, Finnish, Norwegian, and Swedish children achieve around the international average in mathematics, they have very high levels of civic knowledge. Indeed, in this respect, it is Scandinavia (rather than East Asia) where children lead the world.

The PISA Poster Child: A False Promise

Different assessments lead to different conclusions about educational performance. Consequently, it would simply be naive to believe that the outputs of an education system (or any attempt to measure its “quality”) could be compressed into a single (or small number of) scores. Nevertheless, this is exactly how the PISA rankings are often interpreted, not least by politicians and media when the results are released. Hence, with the release of the PISA 2018 results just around the corner, we urge that caution needs to be exercise and how it is vital for readers to look behind the headlines. In particular, calls to copy the education systems of the highest PISA performers should be avoided. Instead, it is worth remembering that the outputs of education systems are multifaceted – and that countries who perform highly in one domain may not do so in another.