Does more money for schools equal better academic performance? Newt Gingrich and the Wall Street Journal say no, relying in large part on the work of Eric Hanushek, an economist at the University of Rochester. His often-cited 1989 review of 38 research studies concluded that "there is no strong or systematic relationship between school expenditures and student performance."

Reducing class size—the most expensive change schools have made in recent years to raise achievement—makes no difference, Hanushek argues. He cites figures for rising education expenditures and falling student performance on tests like the S.A.T. and parts of the National Assessment of Educational Progress (NAEP) as further proof that spending more money on education does not in itself lead to better results.

But Larry Hedges and colleagues at the University of Chicago reached exactly the opposite conclusion last year after a new analysis including many of the same studies Hanushek cites. Using what they considered more sophisticated synthesis methods than Hanushek's, they found that "expenditures are positively related to school outcomes" (italics added) and that class size does make a difference.

Who's right, Hanushek or Hedges? "They're both right, if you look at their data," says Richard Murnane, an economist at the Harvard Graduate School of Education and coauthor, with Frank Levy of M.I.T., of a forthcoming book on schools in a changing economy. Murnane is not being facetious. He is simply pointing out that quantitative research methods can't account for all the complexities of relating school spending to student outcomes. He is also telling us that "Does money matter?"—a hot topic in the current politics of education—is the wrong question.

The Austin Experience

Murnane cites the case of 15 elementary schools in Austin, Texas. Targeted because of poor student performance, each was given $500,000 for each of five years beginning in 1989 to raise achievement. In 13 of the schools, performance and attendance remained extremely low four years later; the other two, the Zavala and Ortega elementary schools, showed dramatic gains.

Using Hanushek's "vote-counting" method of analysis—which basically counts the number of studies that find positive, negative, or no effects, and compares the totals in each category—the conclusion in Austin would be that additional spending has no effect, because a large majority of schools showed no improvement. But in Hedges's "meta-analysis" method, the magnitudes of the effects would also be measured, and the huge gains of the two successful schools would lead to an overall conclusion that increasing expenditures has a positive impact.

The real question, clearly, is not whether money makes a difference on average, but how it makes a difference in the schools that are able to use it effectively. While all the Austin schools used the additional funds to reduce class size, Zavala and Ortega were the only schools that also used smaller classes as an opportunity to change instruction. They adopted new curricula and methods emphasizing more personal attention, and they mainstreamed students with disabilities into regular classes. They also increased parental involvement and improved attendance dramatically by introducing health services to the schools, which serve many families without other access to basic medical care.

The Austin experience helps to explain why researchers have often found—much to the chagrin of teachers—that class size is unimportant. It's not that smaller classes make no difference, says Murnane, but that it's hard for teachers to make the changes that do make a difference. "Even with smaller classes," he says, "it's a rare teacher who can rapidly change the way she teaches. Just reducing class size and hoping that instruction will change is a bad bet."

The Failure of Research

Hanushek and Hedges both concede that their statistical analyses do not account for all the complexities of the money-achievement issue. Yet interest groups on both sides use their data as proof that spending does or does not have a significant impact on school outcomes.

A recent Heritage Foundation report prepared for Congress at the request of Speaker Gingrich uses Hanushek's vote-count data to make the case that "more money and regulation are not the answer." The report says that the opportunity-to-learn standards in the Goals 2000 education reform package prescribe numbers for class size, school materials, and per-pupil spending, despite Hanushek's evidence that these factors have no effect on performance. The Heritage report calls for eliminating the standards, scaling back other aspects of Goals 2000 and the Elementary and Secondary Education Act, and axing the federal Department of Education altogether, an idea that has found favor with Gingrich and other prominent lawmakers.

On the other side, teachers' unions have picked up on Hedges's data, along with other studies that show a positive link between spending and performance, to make a case for increased funding.

But relying on this kind of evidence can lead to gross oversimplifications. "When you compare expenditures and student performance, you don't find a lot of direct effect because of all the complicating factors," says William Cooley of the University of Pittsburgh. "It appears to be a failure of educational research rather than a failure of education."

Cooley's Learning Research and Development Center studies data from every school district in Pennsylvania and reports that three indicators account for more than 60 percent of the variation in average student performance among districts: the percentage of students living in poverty; the percentage of adult residents who are not high school graduates; and the percentage of single-parent homes. These three factors, says Cooley, together gauge the "difficulty of the educational task" for any one school. The more difficult the task—that is, the more poverty, lack of education, and overstressed families in the district—the harder the school must work just to achieve the same level of performance as other schools. Yet, Cooley says, schools with the most difficult tasks also tend to have the least resources.

One implication of Cooley's findings is that one cannot determine by looking at test scores whether a school is good or bad or whether more money does or does not help. "People look at North Dakota and conclude that they've got the best math instruction in the country because they have the highest test scores," he says. "But they fail to take into account the difficulty of the educational task, the fact that they've got a relatively small number of children living in poverty."

The Price of Success

Another implication is that children's education is affected more by what happens outside the school than what goes on inside. "The recognition that education is not just a matter of schooling makes the question of how much money is needed to support it discouragingly complex," says Harold Howe, former U.S. Commissioner of Education and author of Thinking About Our Kids. "Children's success in school is tied to their opportunities in families and communities and to their health, nutrition, housing, recreation, and chances to associate with adults in constructive relationships. How can we ever put a reasonable price tag on all these factors?"

Teacher salaries influence the amount of money schools need to offer high-quality instruction, but their effect varies widely among districts. "The 'Does money matter?' question has a lot to do with the ability to attract and retain good teachers," says Ronald Ferguson of Harvard's Kennedy School of Government, who has studied more than 1,000 school districts in Texas and Alabama. "Teacher salaries may not matter much in the absolute sense, but they matter more in places where teachers have a lot of competing alternatives," he concludes.

Teachers in large metropolitan areas, Ferguson explains, have more job choices. The most talented teachers easily can move to nearby school districts or to jobs in other professions. Thus, districts that pay lower salaries than surrounding districts risk having the least qualified applicants—those with the fewest alternatives. Complicating the issue even further, Ferguson says, is that teachers who are well prepared tend to be attracted to schools in wealthier, higher-status communities. As a result, poor urban districts have to offer even higher salaries to get the teachers that wealthier districts can attract with less money.

"The Texas data confirm that the more attractive schools hire from the front of the queue; the other districts take the teachers who are left over," says Ferguson. "Giving students equal access to high-quality teaching would require low-status districts to pay higher salaries than high-status districts."

Practitioners, often left out of debates on education research, approach the question of how and where money matters from a different perspective, informed not by statistics but by daily experience. Charles Mingo, principal of Chicago's DuSable High School, agrees with Howe's observation that money—or the lack of it—makes a dramatic difference in a child's education far beyond what happens in school. Profiled in Jonathan Kozol's 1992 book Savage Inequalities, DuSable serves an area with a per-capita annual income below $5,000; more than 80 percent of its students are poor. "Poverty affects everything, especially attitudes and aspirations," says Mingo. "These kids need to have some success experiences in school."

Yet DuSable struggles to fund just the sorts of things that might lead to such experiences. The school recently lost its reading program and drama club, and Mingo says it "will never catch up with suburban schools" in computer technology. DuSable also has trouble attracting good teachers because of neighborhood gang problems and salaries that are $20,000 lower than in some suburban schools.

Practitioners are especially leery of research that says spending money to reduce class size doesn't make a difference. "Class size doesn't matter if you don't want students to write," says Vito Perrone, director of Harvard's teacher education programs and a former public school teacher. "It doesn't matter if you don't want them to explore ideas, and if you're only going to use standardized tests to measure what they know."

Test-Score Junkies

In spite of ample evidence that standardized tests are unreliable measures of school quality and can lead to inaccurate conclusions about what works and what doesn't, researchers remain addicted to their use. Test scores carry another danger: news value. Business-oriented publications especially tend to paint the money-achievement connection in a deceptively simple light, using standardized test scores as proof of student failure. The Wall Street Journal in 1993 published a table produced by the Heritage Foundation under the heading "Money Doesn't Help." It compared S.A.T. scores, NAEP math scores, and per-pupil spending levels in the 50 states, showing that some of the lowest-spending states had the highest average test scores and vice versa.

Such a comparison proves nothing, however. Not all high school students take the S.A.T., and a much larger percentage of students take it in some states than in others. Comparing the average score in New Jersey, where 76 percent of the students take the test, with the score in Iowa, where only 5 percent take it (and those tend to be the top students, applying to competitive out-of-state colleges), obviously favors the latter. Moreover, many factors—including home environment and socioeconomic status—have equal or greater impact on S.A.T. scores than does the quality of schools.

Howard Wainer, a researcher for the Educational Testing Service (which administers the S.A.T.), says the Journal's table failed to account for many critical factors, including differences in the cost of living, concentrations of immigrant populations, and differences in spending within states. "It's totally inaccurate to infer a causal implication between spending and S.A.T. scores," he says. He acknowledges that NAEP math scores may provide a more random sample, but points out that these actually show a positive relationship to spending, according to the Heritage Foundation's figures.

The Heritage Foundation itself admits that the research behind the table was not rigorous. "The Journal just picked it up out of our newsletter," says William Lauber, a spokesman for the foundation. "It wasn't very scientific."

Nevertheless, Business Week cited the same "study" in an article headlined "True or False: More Money Buys Better Schools." In it, Stephanie Forest wrote: "American students will underperform those of other industrial countries, despite one of the highest levels of educational spending. Many of the states that spend the most per pupil have the lowest test scores, says a recent Heritage Foundation study."

Eric Hanushek himself disavows the simplistic views of the business press and agrees that "Does money matter?" is the wrong question. "It's a misuse of my research to say that money never matters," he says. Why doesn't he object publicly when prominent publications oversimplify his conclusions in just that way? "That's not my job," says Hanushek.

What Does Matter?

Almost all researchers in the field would agree that educators should focus on how best to use resources to improve student performance. But their advice is often contradictory.

Incentives. Hanushek believes the key lies in incentives: "Right now, schools and teachers that do well have the same expectation for rewards as others, since there's no linkage between costs and performance." He recommends financial and career-building rewards for teachers who produce gains in student achievement and for principals and superintendents who keep achievement up and costs down. Relatively untried programs such as school choice and vouchers might serve as effective incentives for entire schools to improve, he says.

Murnane is more wary of this approach. "Thinking carefully about incentives for teachers and students is absolutely critical," he says, "but it's very hard to design appropriate incentives. I've spent years studying merit pay and am convinced that it's not appropriate. As for other incentives, it's not at all clear what their effect will be."

Class-size thresholds. As Murnane argues, reducing class size will not guarantee improved teaching and should not be considered an end in itself. But smaller classes may well be a prerequisite for the kinds of changes in instruction that do make a difference. Ferguson's research in Texas and Alabama suggests that there is a "threshold" level of about 23 students per class, above which there are measurable negative effects. Here again, however, averages based on quantitative research tell only part of the story.

Teacher pay and school conditions. Susan Moore Johnson of Harvard's Graduate School of Education argues that teacher pay will continue to be an important issue as more career options open up for college-educated women. Administrators also will have to maintain safe and comfortable schools, she says, so that teachers are not lured to other, more pleasant work.

Kozol emphasized the need for safer, better-equipped schools in Savage Inequalities. But Hanushek finds Kozol's emphasis on resources and school atmosphere misguided. "It's almost trivial to say children would have a better chance at learning if they had books to take home," says Hanushek. "It's Kozol-izing the problem. You point to broken windows and ask, 'How could we allow this to happen?' when there's no evidence that any of these things improve achievement."

Changes in professional development. Most teachers place a high value on professional development but are dissatisfied with the opportunities offered to them by their districts. More time and resources for effective learning experiences would increase both teacher satisfaction and student achievement (see "The Old Model of Staff Development Survives In a World Where Everything Else Has Changed,"HEL, January/February 1995).

Reaching Consensus

Virtually all educators and researchers believe that the place where expenditures can make the greatest difference in student learning is in the classroom. But because budgeting and record-keeping practices vary so much from district to district, research has little to say about this area of policymaking. Hanushek, interestingly, believes the charges that central school bureaucracies divert large amounts of money away from classrooms have been exaggerated.

Murnane says the most important changes in education must happen at the interpersonal level. "All education is interaction between human beings," he says. "If that doesn't change, education won't change. It doesn't happen automatically by providing more resources. But making it happen may require more resources.

"I don't think the dollar should lead. First, achieve consensus on goals, then develop a strategy for achieving those goals. Then ask 'What are the resource implications?' But that is not the way policy has typically been made."