Top News Stories
They beg to differ: NAEP, state tests give contrary pictures
Oct 4, 2007 11:24 AM
The good news in newly released math and reading scores from the 2007 National Assessment of Educational Progress (NAEP) is that across the country, many students are improving in reading and math. But the story does not end there.
Results on NAEP, often called “the nation’s report card,” also function as an audit of state test results. This year, as in the previous 2005 round, NAEP scores moved in opposite directions to New York State test results, calling into question just how precise, or how valid, the state tests may be.
On the national tests, which have been consistent in content and scoring since 1990, the 2007 nationwide 4th- and 8th-grade math scores and 4th-grade reading scores were the highest ever. Eighth-grade reading was down a point from its 2002 high.
But results for New York State were mixed. Fourth-grade reading and math scores on NAEP were flat or up slightly, while the state’s own tests showed 4th-graders had lower scores in both subjects over the same 2005-2007 period. Conversely, 8th-grade reading and math scores on the New York NAEP were totally flat from 2005 to 2007 while the state tests showed substantial gains over the same period.
“The issue of comparability between our state tests and NAEP raises questions about the alignment of the state and national tests,” UFT President Randi Weingarten said. “We need to figure out why this happened and what steps can be taken to correct it.”
The nation’s report card
When NAEP results were announced, some commentators dismissed them because the test is given only to a sample of students in New York State. But the sample is representative of city, suburban and rural students, testing experts say. In fact, “In its careful construction over the years, NAEP is more reliable, rigorous and respected than any state’s self-measurement of achievement. There’s a reason it’s called the nation’s report card,” according to Fred Smith, a former Department of Education analyst who is writing a book on testing.
NYU education historian Diane Ravitch called the NAEP results “an embarrassment” for the state. “Just a few months ago, the State Education Department celebrated large gains for 8th-grade students in both reading and math,” she wrote in an editorial. “Unfortunately, they all failed to show up in the NAEP results.”
There were also questions about the timing of gains. U.S. Education Secretary Margaret Spellings used the NAEP results to “prove” that No Child Left Behind, the federal education law, is working and should be reauthorized. But achievement on NAEP actually rose faster before NCLB took effect in 2002. Since 1998, for example, average reading scores for 4th-graders have risen six points, but four of those points were gained before NCLB. And the stagnant 8th-grade scores may indicate that we are teaching reading basics well but failing to instill more advanced comprehension skills.
‘Proficiency’ an issue
The two tests cannot be compared numerically. The scale of the NAEP tests (from 0-500) and New York State standardized achievement tests (200-800) differ. In addition, “proficient” under NAEP represents a higher scale score on that test than a Level 3 represents on the New York State test. But what should be directly comparable is the direction of a gain or loss. That was not the case for the state.
In addition, when a state declares its students proficient in math or reading, that should bear some relation to national proficiency scales. But in 2005, when NAEP reading and math results were last reported for 4th and 8th grades, many states, including New York, were embarrassed to see they had far lower student achievement levels on the national tests than their state tests.
“Basically, the states have embraced low standards and grade inflation,” Ravitch wrote at the time.
Neither the union nor most of its members are opposed to testing. However, in city schools, concern about the state tests has grown, especially as they increasingly become the sole measure of student, teacher and school performance. When New York City 4th-graders surged 10 points on the state ELA exam in 2005, many reading teachers said privately that the test was easier. Teachers routinely talk about “hard” or “easy” test years.
The NAEP results show “just how difficult it is to produce the ‘perfect’ test,” Weingarten said. “We should be building in flexibility and using multiple measures of student achievement rather than attaching so many consequences to a single test score.”
