Popper in the classroom: education pilots with a control group, a proposal for Spain

Every new school year in Spain brings a programme: a plan for reading, a digital plan, a reinforcement plan, a plan against absenteeism. A year or two later someone reports how many schools took part and how many pupils passed. Hardly anyone can say whether those pupils would have passed anyway. Karl Popper had a method for exactly this situation, and Spanish education could adopt it with tools it already has.

Popper's method, in a school

In The Open Society and Its Enemies and The Poverty of Historicism, Popper set piecemeal social engineering against utopian planning. As the Stanford Encyclopedia of Philosophy summarises him, he held that large-scale planning to a blueprint is misconceived, and that reform should tackle social ills one at a time, so that its unintended consequences can be detected and corrected. The entry notes that this mirrors the critical testing of theories in science. A policy is a hypothesis. It deserves a test that could prove it wrong.

In a school system, the cleanest test is a comparison between pupils who get a programme and similar pupils who don't yet, with chance deciding who is in which group. The difference between the two is the programme's effect. Without that comparison, the numbers in the end-of-year report mean very little. Ivàlua, the Catalan public policy evaluation institute, made the point vividly in a 2016 presentation on Barcelona's summer catch-up programme for ESO pupils, Èxit Estiu: after showing participants' results, it asked what those figures said about the programme's impact and answered that they said nothing, because there was no comparison group.

England did it at scale

England built an institution around this idea. The Education Endowment Foundation was set up in 2011 to help raise standards in disadvantaged and challenging schools. By 2020, its head of dissemination and impact told the union Prospect, it had run over 150 randomised controlled trials and published the results, one in two schools in England had taken part in an EEF trial, and around 60% of school leaders referred to its teaching and learning toolkit when planning how to spend the Pupil Premium, the extra money schools get for disadvantaged pupils.

Spain has done it too, once

Spain has its own example. In spring 2021 EsadeEcPol and its partners ran Menttores, online maths tutoring for first and second-year ESO pupils in 18 public and concertada schools in Madrid and Catalonia: three 50-minute sessions a week for eight weeks, in groups of two. The pupils who weren't selected formed the control group. Esade reported that final maths grades rose by 17% against the control group, the equivalent of about six months of learning, that 30% more participants passed the subject and that grade repetition fell by 75%.

Then came the second, equally Popperian step. When regional authorities ran the programme at scale, a 2026 CEP discussion paper found that it raised end-of-year grades by 0.15 standard deviations and test scores by 0.11, about a third of the pilot's effect. The authors call this the voltage drop. It is the most useful number in Spanish education research in years, because it tells every ministry how much of a pilot's promise survives contact with the real system.

The law is there; the method is missing

Spain also has the legal frame. Ley 27/2022 institutionalises the evaluation of public policies in the central administration, provides for evaluation before and after a policy and asks for methodological soundness. In February 2026 Real Decreto 65/2026 approved the statute of the new state agency for public policy evaluation, which designs methods, prepares the government's evaluation plan and carries out evaluations. Neither text mentions control groups. Education, meanwhile, is mostly run by the regions, which is why a central agency alone won't be enough.

The proposal

A Spanish education evidence fund, run jointly by the state evaluation agency, the ministry and the regions that want to take part, with a modest endowment from the state budget and EU funds. Its rules are simple.

First, every new programme above a certain size enters schools through a randomised phase-in. A programme that cannot reach every school in its first year, which is nearly every programme, picks by lottery which schools go first and which go in the second or third wave. J-PAL's guidance on ethics notes that phase-in designs are fairer because no one is permanently denied access, and that researchers must not withhold a service anyone is already entitled to. So the lottery only decides order, and every school gets the programme.

Second, the trial is registered before it starts, with its outcomes named in advance, and the results are published whatever they show. A programme that fails is valuable information.

Third, a programme that works in a pilot is tested again when it scales, so that Spain learns its own voltage drop for each kind of intervention.

Fourth, the fund publishes a Spanish toolkit, in plain language, that any head teacher can use when deciding how to spend support money such as PROA+.

The left's argument that disadvantaged pupils deserve ambitious, well-funded programmes is fully met: everyone gets them, in an order set by lot. The market-liberal demand that public money show results is met too, because the fund closes or redesigns what doesn't work. Popper supplies the piece neither camp brings on its own: the habit of treating every reform as a guess, and building the test into the rollout from day one. Our ladder exercise depends on it, since every rung we propose for a child born poor in 2026 needs a way of showing that it works.