Published by the Australian Education Research Organisation, 10 June 2021
There is a growing appetite in Australia for more evidence-based policymaking in education. In particular, policymakers are often called on to use evidence of ‘what works’ when designing policies to improve teaching in the classroom.
This is a welcome development. Policies based on robust evidence of what works – as opposed to particular interests, ideological beliefs, popular fads or historical precedent (the ‘this is how we have always done it’ way of thinking) – have a greater chance of lifting outcomes for students.
In an ideal world, all education policies would be based on evidence of what works. But, here in the real world, there are many reasons why this can be hard to do. Indeed, evidence-based policymaking can be a difficult, at times frustrating, endeavour.
Government policies are often complex and multilayered. They can be hard to change and even harder to implement. Often the evidence is murky or contradictory. Sometimes researchers genuinely don’t know much at all about what works for a particular issue or context. Still, political imperatives often require action – pronto! Even if researchers do have all the answers for a particular problem, ‘best practice’ can change over time.
In reality, evidence-based policymaking requires an ongoing process of experimentation, evaluation, refinement, improvement and, as uncomfortable as it often is, failure.
What’s more, evidence-based policymaking often involves recognising the limits of the existing evidence base and having the courage to engage in reasoned analysis about the best ways forward, even where ‘hard evidence’ is lacking. This means carefully thinking through tough policy problems and the risks and benefits of different responses.
In fact, the ability to pin down and interrogate a proposed policy’s ‘theory of change’ – including the particular steps or mechanisms that can be expected to connect policy ‘inputs’ to a desired set of ‘outputs’ and ‘outcomes’ – is a key tool in the evidence-based policymaker’s toolkit.
While evidence of ‘what works’ is a great place to start, policymakers can improve policy design by cultivating a broad appetite for ‘evidence’ beyond the existing ‘what works’ research. Four other types of evidence are particularly important.
- Evidence of what doesn’t work. Some of the most effective forms of evidence-based policy making involve refining or scrapping existing policies that encourage practices that are no longer supported by research evidence. This can involve tough decisions, but the payoffs can be significant.
- Evidence of current practices in schools and classrooms. Knowing what is currently happening inside schools (opening up the ‘black box’) is essential for policymakers to understand the types of challenges teachers and students face, prioritise among these, and design sensible, well-targeted policies in response.
- Evidence about policy implementation. It is one thing to be familiar with the ‘what works’ evidence base. It is quite another to know how to design and implement policies that ensure evidence is put into practice in schools. Incorporating what is already known about effective implementation approaches is essential for good policy design. Policymakers can also build this evidence base by evaluating implementation processes, not just policy outcomes. Policies that have no chance of being implemented help no one.
- ‘Missing’ evidence. Given the limitations of the ‘what works’ evidence base, policymakers should think hard about the questions they most want answered and commit to finding out more. A process of disciplined innovation, including small scale pilots that are evaluated rigorously, can help fill in the gaps in our knowledge.
Committing to evidence-based policymaking doesn’t guarantee we can or will solve all of the hard problems in school education. But it increases the odds that we will get a little closer to achieving the things we really value. That makes it worth the effort.