Insight
Is there really a charter ‘effect’?
Oct 15, 2009 12:24 PM
Mayor Bloomberg recently called for policy changes that would allow a rapid expansion of charter schools in the city.
He backed that call by citing a new report by Stanford University Professor Caroline Hoxby, published, fortuitously, just a few days earlier. Hoxby’s report found that students in New York City charters performed a few points better on state tests than those who entered charter lotteries but were not selected (the “lotteried out”).
In Hoxby’s view, this puts to eternal rest opponents’ arguments that many charters do better with students because they have better students to start with. Hoxby argues that the “lotteried in” and “lotteried out” groups in her study were substantially the same demographically, so hers was an apples-to-apples comparison. Thus, she concludes the reason for the better performance of the lotteried-in students is a rather magical charter “effect.”
Maybe.
A UFT analysis just found that city charters have significantly lower levels of free-lunch students. Children from families whose income is no more than 120 percent of the federal poverty level (free lunch only, not free and reduced-price lunch) are much less likely to be in charter schools.
Hoxby is a controversial scholar known as a promoter of charter schools. Her report followed the June publication of another Stanford University study, this one by its Center for Research on Education Outcomes (CREDO). In its study of 16 states, CREDO found the opposite: charter school students on average slightly underperformed their traditional school peers in reading and math.
Of 2,403 charter schols in CREDO’s study, 46 percent had math gains that were statistically indistinguishable from the average for students in traditional public schools. Seventeen percent of charters did better in math growth and 37 percent did worse.
That study followed reports by the RAND Corporation in 2005 and the National Center for Education Statistics in 2006 that also found charters not doing quite as well on average as district public schools.
The Harlem-Scarsdale gap
So in Hoxby’s new report, the test score advantage for New York’s charter students is about 3 to 5 points on a 325-point scale for reading and math. She finds a charter advantage in social studies and science as well, but at a level of statistical reliability (p-value = 0.15 for you math teachers) that is not generally accepted by researchers.
And Hoxby overplays her results. She claims that students attending charters could close about 86 percent of what she calls the “Scarsdale-Harlem achievement gap” in math and 66 percent of the gap in English.
She does this by making a projection. She takes the average annual gain for all students and assumes each student would make that gain every year, from kindergarten through 8th grade.
This is stretching the data too far. As teachers know, most students do not sustain the same gain year after year for eight years. And in fact, very few students in the Hoxby study even attended charters that long.
Besides, it’s not just a few test-score points that stand between Harlem students and their Scarsdale counterparts. The resources that Harlem students start with in life are a far cry from what a Scarsdale youngster gets as his birthright.
Closing the gaps
Many city charters run much longer school days and school years than traditional public schools, likely an important factor in their test score gains. Some, like those in the widely admired Harlem Children’s Zone, couple education with “wrap-around” services that make a big difference.
That model should be expanded, as Bloomberg has proposed. The Harlem Children’s Zone has found ways to level the playing field for students by providing children in its area with health and mental health services, family supports and community building to create the resources that open children’s worlds.
So the additional education services that the mayor is advocating for charter schools can indeed make a significant difference. Which is why they should be made available to all students in the city.
