What these test results reveal about Australia’s education system

Last week, the OECD released the results of the Programme for International Student Assessment, better known as PISA. Every three years, hundreds of thousands of 15-year-old students from across the world sit tests in maths, science and reading. But what do the results show for Australia? And are our students doing better, worse, or about the same?

Listen to Jordana Hunter, Education Program Director, and Nick Parkinson, Senior Associate, in conversation with host Kat Clay about the test and what the results reveal about Australia’s education system.

Transcript

Kat Clay: Last week, the OECD released the results of the program for international student assessment, better known as PISA. Every three years, hundreds of thousands of 15-year-old students from across the world sit tests in maths, science and reading. But what do the results show for Australia? And are our students doing better, worse, or about the same? Here to talk to me about how Australia has performed and how we can lift our performance are program director of Grattan’s education team, Jordana Hunter and senior associate in the team, Nick Parkinson. So Jordana, I’m really interested in what is PISA and, and why should we care about this?

Jordana Hunter: Yeah, look, PISA is kind of a big deal. It’s one of the few large scale assessments, we have access to in Australia where we can benchmark what Australian students know compared to students around the world and even compared to Australian students in PISA assessments.

So the 2022 assessments, which were published on Tuesday last week, involved 81 countries around the world and almost 700,000 students at those tests In Australia, the tests are based on a sample. So not every student sits them. We had about 13,000 students sit those tests across 750 schools, and that gives us a really good picture into what our 15 year olds know and can do with that knowledge. it, it does help us really understand how our education system is traveling, which is, which is really important for policymakers and it’s important for parents and everyone with an interest in the quality of school education. There are also some great surveys in PISA that get a picture from teachers and from students about school life.

So everything from levels of disruption in the classroom, the amount of resourcing that schools have got access to, how teachers feel about their pedagogy, how students feel about teaching practices. So it’s a great picture and it really, as I said, gives us that opportunity to get a sense of where we’re tracking compared to the rest of the world.

Kat Clay: Yeah, and we’ll have you back on the podcast to talk about some of those results when they come out, about how people are feeling about the schools, you know, in the classroom later. But what I really wanna know and what everyone wants to know is how does Australia stack up to the rest of the world? How are we doing here?

Nick Parkinson: We’re really pretty stable. So we’ve, we’ve weathered the COVID storm all right. The results from 2022 are neck and neck with the results from 2018 and 2015.

Now that’s in the context of a large backwards slide across the OECD and across countries more broadly. When we look at PISA Poster Child Finland, for example, its results have plummeted and now it’s reading results are worse than Australia’s.

So in that context of a post pandemic backwards slide. Australia has held its ground well, and because of that, teachers, students, and families ought to be commended. What I would note though in, in your question about how do we stack up internationally is that other countries, including some really high performing countries, also held their ground or indeed improved. In maths for example, Chinese Taipei. They improved and they’re a real high performer. And Singapore and Japan also maths powerhouses in PISA held their ground just like Australia did. In Science Canada they held steady and they’re outperforming Australia and in reading Ireland also held steady and they’re also a strong performer.

Kat Clay: So Jordana, does this mean Australia did reasonably well at maintaining a good level of education during the pandemic?

Jordana Hunter: I certainly looks like Australia weathered the storm, in terms of those pandemic disruptions to schools. Pretty well, and I think Australia has also focused on catch-up support for students that fell behind. So particularly in Victoria and New South Wales, there’s been a lot of investment in catch-up tutoring to help those students that really did struggle to keep up with their learning when schools were closed. So I think that is good news, and I think it is a testament to the hard work of teachers and parents and students themselves really to get through that. That difficult period. It’s not to say that we’re unscathed. there’s certainly some negative consequences of that pandemic for schools, and students.

And we see that in the wellbeing data, particularly around mental health. And we also see that in the attendance data. think compared to some other countries, it does look like we’ve held up better. If we compare our results to those internationally in the mathematics domain, there were 30 odd countries that either held up well like we did or improved. So we’re in good company. But you know, certainly it’s a much better story than one, countries were facing When they opened those results and saw that they’d slid back considerably.

Kat Clay: And I mean, how are we keeping up with those top performers?

Jordana Hunter: Well, look, I think this is an important point to note because while Australia has held steady since the 2015 results and the 2018 results, we are not actually doing a great job of catching up to those top performers. Singapore is really the standout country in PISA. They, they are really significantly above Australia.

And just to put that in context, in reading and science, their 15 year olds on average are about three years ahead of Australian 15 year olds. In mathematics, it’s more like four years. So when you think about that 15 year olds in Singapore can do a lot more stuff in maths and science than kids in Australia. And they can also, they’re also more skilled readers so they can read a text and, and understand that text and apply the information from that text a lot more effectively than Australian 15 year olds. We want Australian children to be competitive in those international, employment markets. We want them to be generators of innovation in Australia. You know, human capital is important. it’s also important for individual flourishing. I, I think it’s clear that we’ve got some more work to do in that space. And I might just add as well, Kat, if we think about how we’ve gone over time we’re well off where we were back when PISA assessments first started in 2000 and 2003. So 15 year olds today they’re performing at a level that’s about a year or more behind where they were back in 2000 and 2003. So we’re actually going backwards, compared to our own performance. and that is something that we really need to turn around.

Kat Clay: I think it’s really interesting then to kind of dive into the Australian data because that’s where we are gonna find the really interesting information relevant for, you know, education policy here. Nick, what did you find having a look at this data?

Nick Parkinson: Well, it’s really interesting, Kat, I think we talked before about how, in the context of the backwards slide and general tendency post pandemic to to have a drop off. Australia’s been, I think, relatively unscathed to the words we used before, and, and that’s true at an aggregate level. But when you start looking at some of this data, there are some concerning pictures.

About half of Australians aren’t meeting the national proficient standard in maths. So that’s a standard that we set for our 15 year olds. We say that this is what a 15-year-old should be able to do to, to be able to get through life with the maths that they need. Now in science and reading about two in five, just over two in five fall short of the national proficient standards we’ve set for those two tests.

So that’s a lot of 15 year olds around Australia. Who are really struggling at 15 to do the things that we want them to be able to do.

And this speaks to a broader tendency when we think about that, that question before of how is Australia stacking up? How are we tracking and what does PISA tell us? We can kind of see a tale of two Australias. So in mathematics, for example, students in regional schools and in in regional Australia, they did have a significant drop off since the 2018 piece of tests in their maths performance.

And, and that is something that could, should concern us. We don’t want these gaps to grow any wider. Indeed, when we do look at the variation in achievement, for example, between the top and lowest performers in Australia. That that spread is wider than it’s been, and that spread is also wider than the OECD average.

That is a concern and we really ought to be thinking as a country about what can we do to make sure that we not just have a world-class system that’s competing with those top countries, but a system that is fair and as equitable as possible so that even those students who are starting well behind to gave it having, you know, a good shot at being able to do all the things we’d like a 15-year-old to be able to do.

Kat Clay: Can I ask Nick? I mean, why do you think that is? Has there been any research on why the, there’s a decline or is it it’s just such new data we need to investigate more?

Nick Parkinson: I think that there, there’s definitely room to do more investigation. What we do know is that students had variable experiences throughout the pandemic.

Pisa has asked students some questions about, for example, the extent to which they felt someone was available to help. And there are a concerning number of Australian students who report that they weren’t able to get the help they need. So what we see is, a bit of a variation in the extent to which we’ve been able to support students to get back on track with their learning.

And when you look at some of this data, it might be pointing towards that being more challenging for students in context that are disadvantaged.

Kat Clay: Yeah, and I suspect, you know, access to technology and internet probably also has a role to play there too. But I’ll be very interested to see, you know, when we dig into that information and those, those survey results, what comes out of that, especially in regional areas.

And so we’ve talked about the difference between regional and urban areas, but what about the division between the states and territories? Was there anything that came out there?

Nick Parkinson: This is a really interesting question, Kat, and I’ll focus on maths because that has been the topic of interest for this PISA cycle. So each PISA test changes, between the, the test that it goes kind of in depth and does a deep dive on. When we think about the math performance across states and territories.

On a quick glance, there is a lot of room. For improvement in each state and territory, even at just an absolute achievement level. So across all states and territories, we’re still talking, you know, at least 30% of students not meeting the National Proficient Standard. When you strip out socioeconomic advantage, differences between the states and territories are almost indistinguishable or negligible. This points to to room for improvement in every state and territory in mass achievement. And the same also looks true when we are looking at that top level result for science and reading

Kat Clay: Well, let’s dive a little bit more into the detailed kind of areas that they’ve tested. I mean, Jordana, you and the education team have been producing a lot of work at the moment on reading and why it is so important. and for me it’s been an eye-opener because, you know, we think, we think on that granular level that these kids are doing tests at school, okay?

The results middling to not great. What we don’t really often think about is how that all has flow on effect for people as they go on in life and how important reading is. So I was wondering if you could talk a little bit to us about what Australia can do to lift its reading performance and just emphasize kind of why that is so important.

Jordana Hunter: I mean, reading really is The gateway skill, if you struggle to read, not only do you struggle to access, you know, a broad and rich curriculum in all other subject areas, including mathematics, are really gonna struggle, coping in life and taking advantage of, you know, good employment opportunities. able to interpret healthcare advice, make up good financial planning decisions. I mean, reading really does underpin every aspect of life in, in a modern society. Australia does have some work to do here and I think the PISA results show that the recent NAPLAN results also show that, there is a lot of room for improvement. Yeah, we’ve heard a lot about the debate about, you know, phonics in particular in early primary years, that is really important.

I think there is a lot of work to do to lift, the confidence and the capacity of teachers to deliver a really structured literacy program in those early years of school that does provide a lot of practice for students in a sequential way of understanding the, the alphabetic code and, and sort of how to decode words, break words down. Phonics is a key part of that. I think the other bit though that we haven’t fully wrapped our heads around in Australia is the really critical role of vocabulary and background knowledge for students as they move through school. We know as students, get older and they’ve transitioned from, to read, to reading, to learn, their ability to really comprehend a text, understand what a text is saying largely comes down to their knowledge of vocabulary and background knowledge. For a lot of our students when they sit PISA at 15. You know, they’ve been building that knowledge bank at home, outside school. for those students who perhaps come from more disadvantaged backgrounds where they haven’t had as much access to, know, knowledge rich environments in the, in the home, they’re really reliant on those 10 years of school. I do think we have to get better at being more systematic at building background knowledge. This is hard work for teachers because designing really knowledge rich well-sequenced, curriculum programs for their classroom is a huge amount of work. Grattan’s written about that extensively in our Lesson lottery report, which we published at the end of 2022.

Kat Clay: Because I can imagine things like phonics is gonna require a lot of curriculum resources and we’ve already, we, as we know from our research, you know, teachers are very time poor at the moment. So having those resources handy would be incredible. What is government’s role here in improving reading?

Jordana Hunter: I’d be recommending government does, you know, some more of the steering and maybe a little bit less of the rowing. We have some great, providers of curriculum materials out there. There’s some great, not-for-profits. There’s some great commercial providers, some schools are doing some great work as well.

In this space, I think there’s a real opportunity for government to support a robust quality assurance mechanism. That can actually, you know, put the blowtorch on some of these materials. make sure that they are curriculum, curriculum aligned, that they’re well sequenced, that they’re being road tested for the classroom so that schools and teachers have got the confidence, to select materials that they think will work in their context and adapt them if they want to. There’s probably a need, I think for a really significant step up in the training that teachers have. I think it’s clear that not all teachers are coming out of initial teacher education with the skills that they need to teach reading in the early years as well as they could be. I’d also like to see governments investing more in, supporting schools to do universal screening so that they identify students as early as possible if they’re struggling and provide really robust intervention support.

We know we can catch students up, we can catch up the vast majority of students that fall behind as long as we know early. And finally, a national phonics. . Screening test, in year one would help build that national picture of how students are going, and could also give parents reassurance that their students, their children are on track. Really, the year three NAPLAN results are a bit late. We should know well before then at a national level how students are traveling with those foundational reading skills.

Kat Clay: Turning to maths and science, Nick, can we be improving Australian tuition, the, in those areas? are we gonna be doing more work on that as well?

Track 1: Maths is definitely something that’s on Grattan’s mind. I think before I get to some of the things that Grattan will be looking into when we’re thinking about how to lift maths performance, it’s worth just reminding ourselves of what is PISA like as a test. ’cause that speaks to a lot of, a lot of the challenges and a lot of what Jordana and I are talking about.

PISA is really asking students to apply knowledge that they’ve accumulated through 10 years of schooling. When you look at the questions for reading, for example, students might be comparing, two conservation programs and actually be asked to justify, a choice made between those two conservation programs in the way they’re being designed.

might have some stimulus material in science, uh, involves looking at some charts of life expectancy and then doing some more open-ended questions on those charts. What I’m really saying through this is that the complexity of what students are trying to do in PISA relies on this really hierarchical foundation of knowledge that they have built through great teaching day in, day out from the start of school.

What we then need to be thinking about in terms of 20 years from now, how can Australia be higher up in PISA? Indeed, even before then when we’ve got, you know, today’s prep sitting PISA is making sure that they’ve developed the skills and their knowledge they need so they can actually do that really high order stuff.

So some of the thinking that we’ve been doing in mathematics, and as I mentioned it’s early days, so we will have more to say about this in the future, is thinking about how can we build a really strong foundation in primary school so that students kind of automatic with all the things we need them to be automatic in, in maths so that they can start doing that really challenging, heavy lifting that, that they need to do in PISA.

But they also will need to do kind of beyond school, both in the secondary years university and in life more general.

Kat Clay: I’m really curious, like what kinds of questions, maths questions do they ask on the PISA test? Do you have an example of one?

Nick Parkinson: The kind of things that they might be asked is a multi-part question where . They’re given a scenario and then asked questions which relate to that scenario. Some of those questions will be multiple choice and some will be open answer or, or short answer. An example might be, students have an array of die and they have to look at the, you know, sum up the total numbers on those die and then figure out what is the pattern.

 It’s really asking students to use their mathematical reasoning, and other, other skills and maths that, are beyond basic maths facts. Now sometimes there people infer from that that, oh, well PISA is all about applications. So, um, we can do away with maths facts.

We’ve got calculators and everything now. That’s not quite how maths works. You really need to know, those maths facts so that you can actually start being able to do things like applying your maths.

Kat Clay: So, I mean, we’ve talked a little bit about how government can improve, reading in Australia. What about maths and science? Is there anything that we could potentially recommend right now?

Nick Parkinson: I wouldn’t wanna hedge my bets on a recommendation right now, but I can tell you some of the things that Grattan are starting to look into. So one of the things is that maths is actually a really, really challenging thing to teach, particularly if you’re quite good at it because you have to try to take yourself back in primary school to be able to think about what does an early someone learning maths need to know?

You know, what does it actually mean to divide a fraction by a fraction? One of the things that’s really important about maths is that that maths content knowledge and that maths pedagogical knowledge. So, what does it actually mean to know maths and what’s the best way to teach it? And what we know is in primary school actually most of the people teaching maths may not have studied maths themselves, and, done a general primary teaching degree, maybe with a few math selectives.

So one of the things that we’re thinking about, and that is a feature of other systems across the world, which do well at maths, is what might be a role for training up and deploying primary math specialists. And some jurisdictions in Australia have trialed this before The other things that are really important in terms of improving maths and science teaching are just like Jordana mentioned with reading, developing and giving teachers access to really high quality curriculum materials, particularly for those teachers who might be teaching maths and science but are completely out of field for that subject.

That is, they themselves have never studied it, but they’re in a school which doesn’t have a specialist math teacher or timetabling has worked out that they’re taking that subject. Three other quick things that are on Grattan’s mind and that we’re looking more into and starting to look at the evidence base for them.

what does really great curriculum specific professional learning look like? What role might they be for that? What’s the role of catch-up tutoring in helping students who have fallen behind in maths and science, and what’s the best way to arrange and organize that tutoring? And finally, there are some high quality technological innovations out there, such as personalized digital tutors that can give students an extra dose of learning, particularly in the home.

and that can be at their point of need, whether that’s helping extend and challenge them or helping them catch up. In addition to the teacher who’s in their class and teaching them during the day.

Kat Clay: Thank you so much Nick and Jordana, for discussing and breaking down those recent PISA results. What I’m really interested in is going forwards, you know, finding out more about what students have said about life in the classroom and also just hearing the research that you’ve got on the burner in terms of reading, maths and science and we’ll no doubt, have you back on the podcast in the months to come. If you’d like to read more about our research and our articles on education and reading, please visit our website grattan.edu.au. We are a not-for-profit organization and we rely on donations from lovely people like yourselves. If you enjoyed this podcast and you want to see our research and our podcast please donate at grattan.edu.au/donate. always, do take care and thanks so much for listening.