Influential studies over the past few decades by Stanford’s Eric Hanushek and others have concluded that there is scant connection between spending and student achievement. And many policymakers, including U.S. Education Secretary Betsy DeVos, have used the research to justify lower funding for public education or to promote private school choice programs. But in a new analysis of both the older research and new work using new methods, Northwestern University researcher and FutureEd adviser Kirabo Jackson contends that the conventional wisdom on school spending is wrong, that money does influence student achievement. He spoke with FutureEd recently about his findings.
Many researchers have argued that increased education spending doesn’t translate to higher student achievement. Your new work suggests otherwise. Why?
Let me start out by saying that it’s pretty clear that money does matter. But it’s not a panacea. And the question of whether money matters is separate from whether money can matter more if we spend it better. Are there ways in which we can figure out those areas where we’re going to get the biggest return on investment for our money? We should absolutely identify those areas.
Now to your question, it’s not true that previously there was little evidence that money mattered. A narrative had emerged that, by and large, these early studies do not provide consistent evidence that money matters. But it turns out that the data doesn’t justify that conclusion.
Can you explain that?
Eric Hanushek is famous for looking at a whole bunch of studies and concluding that if money matters we should see at least half of the studies on the effect of school spending on student outcomes having positive, statistically significant impacts. That sounds reasonable, but that’s actually not the way statistics works.
Why is that wrong?
By way of background, in statistics we say that a study is statistically significant if there is less than a 5 percent chance that the result occurred at random (just due to having a lucky draw of the sample). As such, if there were no impact at all then we would expect, roughly speaking, just by random chance, that 5 percent of the studies would show a level of significant improvement. So 5 percent is the benchmark we should be using, not 50 percent. About 40 percent of the earlier studies found a significant improvement. So the basic frame—that the old studies didn’t find evidence that money matter—is false. Having said that, new studies provide much more compelling evidence that money does matter than this older literature.
Why?
Most of the older studies done prior to about 1995 were based largely on correlations, rather than any type of policy experiment. There are many reasons why some jurisdictions may spend more money on a school than others that have nothing to do with the impact of school spending per se. They might be wealthier areas. They might be areas that have policies that promote higher student outcomes. So you really can’t make a causal inference based on data that are just comparing places that happen to spend more or happen to spend less.
What changed after 1995?
There was a shift in empirical microeconomics and in policy research in general, toward looking for what we call natural experiments. What researchers are trying to do is to find situations in the real world that approximate a randomized controlled trial or randomized experiment.
Does school finance reform, the effort to have more equitable funding, fit into that?
Yes. School finance reforms are one example of that. They provide something that approximates a kind of experiment. Of course, it is not entirely random. But it is a scenario where we know when a state supreme court decides they have to rewrite the rules of how schools are funded. We know the day when that happens, when the existing system is determined to be unconstitutional. Suddenly school spending is going to change for reasons that are unforeseen by school districts and that are not under the control of school districts or parents.
Some districts are going to get more money and some districts may lose money. So that basically is a scenario where you can look at the changes that occur within school districts at the time of the school finance reforms. And you can basically figure out what happens when a school district gets a sudden inflow of cash. Do kids who are in school at that time period have better outcomes?
And the answer to that is: Yes. Almost every single study that looked at achievement in the wake of school finance reform has found that if you are a child in school, and suddenly because of something that happened at the state level, your school district gets more money on a per-child basis, your outcomes are going to be better than, say, your older siblings, who were in school before the infusion of cash occurred.
The same is true for referenda. Often local areas vote to increase school spending. If you look at an area that is split close to 50-50, whether the referendum actually passes with 50 percent plus one vote or fails with 50 percent minus one vote, those areas are super, super, super similar. They have very similar dispositions toward wanting to increase school spending. But in one place it narrowly wins, and in another it narrowly misses. That produces big changes in school spending that you can use as an experiment.
And your recent study found cuts in school spending can provide a similar point of comparison.
That’s another kind of a natural experiment. In the wake of the Great Recession a lot of individuals lost spending power, consumption was down. People spent less money, and a lot of people lost their jobs. In general, state taxes fell pretty rapidly right after the recession. Some states fund schools to a large extent based on local property taxes. Other states rely heavily on state taxes. Others rely more heavily on federal funding. There are differences. So you may have two states, both of whom were exposed to the Great Recession, but the states that relied most heavily on state taxes were the ones that saw pretty sudden reductions in their per-pupil spending.
[Read More: How Do School Spending Cuts Affect Achievement?]
Children who were in those schools during the recession, compared to those that were in schools before the recession, saw per-pupil spending levels drop—and they saw their test scores and high school graduation rates drop. Using this approach, myself and coauthors find that a 10 percent school spending cut over the previous four years reduced test scores by about 6 percent of a standard deviation and graduation rates by about 1.5 percentage points.
In your broader review of the research, what is the impact of spending changes on disadvantaged students?
Some studies find positive impacts for disadvantaged students, including some of my work using school finance reforms. But this is not always the case. So I would say that the evidence on this is mixed. But it tends to suggest that less-advantaged children benefit more.
You mention one study where the presence of strong unions seems to lead to better outcomes. Do you have a sense of why that happened?
This study was also relying, to some extent, on school-financial reforms. What the researchers basically documented is that in areas where you have very weak unions, the additional money that came from the state ended up being used to give residents in the local areas a tax break. It was only in areas where you had relatively strong unions that the additional money from the state actually went into spending in the classroom.
As a result, students in those districts had improved outcomes. Areas that had weak unions used the money to basically provide tax relief and test scores did not improve very much.
So context is important. What can we learn from studies that consider multiple states?
There are many recent and credible school spending studies that use national data or multi-state datasets. these studies are going to give you a sense of whether on average money matters. And about 90 percent of studies that look across the entire nation find positive and significant impact of total spending on student outcomes. So that tells you that, on average, money absolutely matters, but it does not necessarily mean money matters in every context, in all settings, in all school districts.
You found mixed evidence among various studies on the impact of federal Title I spending on achievement.
There are two things to say about Title I, and they relate closely to what I was saying about unions. It has actually been pretty well documented that when the federal government provides additional Title I funding to school districts, the money does not always make it into the classroom. What often happens is it gets basically offset or crowded out. When districts get additional money from Title I, they often reduce taxes and spend less local money on schools. That’s one explanation from why we tend to see pretty small impacts of Title I funding.
But there are a couple of studies that have found that federal spending doesn’t always supplant local money and does make it to the classroom. Also, studies that look at Title I using multi-state analyses tend to find positive impacts.
Did you find there were certain kinds of spending that were most effective in improving achievement?
One pattern does emerge. Studies that look at increasing the budget without constraints almost uniformly find positive impacts. It seems as though just giving schools more money to spend however they please tends to improve outcomes, as counter-intuitive as that may seem.
The other pattern that emerged was that money that is spent on capital spending, such as new school construction, tends to be less likely to affect achievement.
If you can identify areas where spending makes a big difference in achievement, it makes sense to focus spending in that area. Spending on new and better textbooks, for example, can have a massive impact. But if you are prescribing spending and get it wrong, then you may actually have no impact at all. So, in some cases, you may be better off just giving money unconditionally to districts and have them decide how to spend it.
View the List of Recent Studies
Photo courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action.