Insight
What works? Curriculum mostly
Nov 12, 2009 1:55 PM
Where should New York City and the rest of the nation spend precious education dollars, especially now that we don’t have so many?
A short but important new paper suggests that we have been investing in many expensive “solutions,” like charter schools or merit pay, while overlooking the material that every teacher works with every day in every classroom — the curriculum.
Research shows that the effect of good curriculum on student achievement is much larger than the effects of most other popular policy tools, according to Grover “Russ” Whitehurst of the Brookings Institution. “Further, in many cases they are a free good,” he writes in the paper “Don’t Forget Curriculum.” A good curriculum generally costs no more than a bad one, so “leaving curriculum reform off the table or giving it a very small place makes no sense,” he writes.
Whitehurst’s paper, published in October, comes as the federal Department of Education is getting ready to award $4.35 billion in Race to the Top grants to states that propose the most innovative and effective education reforms. Here is a pot of money not likely to come again, and it is one that could help transform the education landscape.
What works
Both President Barack Obama and former President George W. Bush called for spending education money on “what works” — that is, research-proven strategies. But there is widespread concern that the education department is ignoring the research in its approach to these grants. Despite limited evidence of their effectiveness as strategies, Education Secretary Arne Duncan has made it quite clear that increasing the number of charter schools and evaluating teachers based on student test scores will give states a competitive edge for Race to the Top money.
Whitehurst’s modest paper challenges these priorities. He systematically reviewed studies of the effects of five education interventions, including charter schools, performance pay, early childhood programs, content standards and curriculum. He found only one of them has a major transformative effect, and that is curriculum.
(Whitehurst, so you know, has no particular horse in the race. He is the recently retired director of the Institute of Education Sciences at the U.S. Department of Education, an institute admired for its rigorous focus on what the research says in the face of sometimes powerful political or commercial pressures.)
The results of Whitehurst’s review
Here are some of Whitehurst’s key findings:
- Studies have generally found very small differences in achievement between students in most charter schools and those in other public schools. Whitehurst notes more positive findings for charters in studies of “oversubscribed” charters where students must win a lottery to attend. In those, one recent study shows an “effect size” of 0.09 in math and 0.06 in reading. But that is a small effect, and typical charter school outcomes are closer to zero.
- (Effect sizes are a statistical way to measure the strength of an educational input on a student outcome. The closer to 1.0, the stronger the effect.)
- Upgrading the teacher work force, by trying to attract higher-scoring teachers and offering alternative pathways to certification, has some research support. A study of Teach for America recruits found a 0.15 effect on student math scores but no effect in reading, he says. Merit pay also showed some positive effects on achievement, but the strongest finding (a 0.15 effect) was in India and involved substantial bonuses.
- Early childhood programs, especially those like the small but famous demonstration projects — the Perry Preschool Program in Michigan and the Abecedarian Program in North Carolina — have strong achievement outcomes in reading. These programs had extensive resources and very small student-teacher ratios. But large-scale federal programs like Head Start, with many fewer resources per child, have shown much smaller measurable effects.
Content standards are a major focus of state education departments, including New York’s. The National Governors Association has launched a project to develop a “common core” of standards, and Education Secretary Duncan has made states’ participation in the Common Core initiative a prerequisite for Race to the Top funding.
Here Whitehurst conducted his own study. He looked at 4th-grade math scores on the National Assessment of Educational Progress and the quality ratings assigned to states’ content standards by the Fordham Foundation and the American Federation of Teachers (separately). He found very small effects even for those states whose standards were rated high by both Fordham and the AFT. And there was no predictable pattern. For example, Massachusetts’ standards were highly rated by both Fordham and the AFT and it has high NAEP scores. But New Jersey had low content standards on both Fordham’s and AFT’s scales, yet it scores about the same as Massachusetts on NAEP.
“The lack of evidence that better content standards enhance student achievement is remarkable given the level of investment in this policy and the high hopes attached to it,” writes Whitehurst.
What about curriculum?
We hear relatively little from education leaders about the importance of using the best curricula out there. Yet the IES has evaluated many of them, and according to Whitehurst’s work, curriculum quality matters — a whole lot.
For example, the IES looked at four elementary school math curricula, comparing student math achievement at the start and end of 1st grade. Students’ percentile ranking was 9 to 12 points higher at the end of the school year for those who used Math Expressions and Saxon Math compared with those using Investigations in Numbers, Data and Space or Scott Foresman-Addison Wesley Mathematics.
An IES study of preschoolers found that a curriculum of DLM Early Childhood Express supplemented with Open Court Reading had an effect size of 0.76 in early reading and 0.48 in vocabulary.
Several beginning reading programs produced effect sizes above 0.80 for alphabetics (sound-letter correspondence). These are astronomical effect sizes for young children from low-income families.
Two programs designed to reduce dropping out, Accelerated Middle Schools and Check and Connect, had a 1.0 effect — in other words, a perfect result.
Whitehurst’s table shows the effect sizes for the various programs he compares.
Anyone interested in “doing what works for the kids” should pay attention to this table. The bang for the buck is pretty clear.
