Insight
Moving Students forward — how do you measure that?
Sep 20, 2007 1:31 PM
Starting with the launch of No Child Left Behind (NCLB) in 2002, the hoopla surrounding test results has climbed to a fever pitch. In districts such as New York City, where some half of its heavily poor and minority students have yet to meet state standards, elected and appointed officials and editorial writers loudly condemn the schools: Students aren't trying, parents don't care and teachers aren't teaching right.
All this time, teachers have been shouting to be heard: These tests measure a student at a single point in time; they are harsh and they fail to give students and schools credit for hard work and progress; many kids, especially in New York City, overcome huge education deficits to get better scores, but still don't "pass."
What do these punitive laws have to offer such students? And what educator wants to work in a school that is constantly branded as failing no matter what progress it makes?
Progress as evidence
Well, better late than never, the nature of assessment is changing — again. Not far enough, not fast enough, but it has started.
Legislators on the left and right are urging that NCLB be amended to give students more credit for progress when the law is reauthorized next year.
New York State is putting in a sophisticated new data system so it can track student progress.
And closer to home, the Department of Education's new School Progress Reports (notice the name — they are not called School Report Cards) are to give more weight to student progress than to absolute test performance in evaluating schools. Reaching Level 3 is not the only way to get credit. Those reports are due to be released in a couple of weeks.
These developments represent a big step forward in assessment. The achievement benchmarks on the state tests for Grades 3-8 — Levels 1, 2, 3 and 4 — are more closely associated with student socioeconomic status than anything else. Evaluating students and schools based on progress — not just whether they made it to Level 3, but whether they progressed from a year ago — is a fairer way to judge success.
[Note: Details, below, of the new progress reports focus on the elementary and middle school measures. High schools are graded similarly but use different measures — mostly graduation rates and Regents passing rates.]
Measuring progress
Measuring progress is a pretty simple concept, though the calculations can get complex. Imagine a child in 4th grade scores at a mid-Level 2 on the 4th grade ELA. That child should score at least at a mid-Level 2 on the 5th-grade ELA Ń that is, he should make a year of progress during a year of school work. If he does, he gets credit for that achievement. On the new School Progress Reports, the average progress of all the students in a school, whether they meet standards or not, accounts for 55 percent of a school's final grade.
(The exception to this is Level 1 students. They do not get credit for remaining in Level 1. They must progress to at least a low Level 2.)
If that same mid-Level 2 student in 4th grade scores in the high Level 2 range in 5th grade, though, he gets even more credit. Again his extra progress is averaged together with the students in the rest of the school (including children progressing from low Level 1s to high Level 1s) and their collective gains or losses factor into the score.
Finally, there is additional weighting for the change in proficiency of the lowest-performing one-third of students. This pushes schools to pay more attention to struggling students who may otherwise be neglected amid the pressure to get every child to standards.
But meeting standards still matters, as it should. Another 30 percent of the school grade is based on the percentage of students actually meeting the standards, meaning scoring at Level 3 or 4, or in high schools, graduating or passing Regents. So schools have an incentive to not just move students along but move them up to a benchmark.
The final 15 percent of the schoolŐs grade is based on non-test-score ŇenvironmentalÓ factors, including attendance and feedback that parents, students and teachers give about the school in the new Learning Environment Surveys [see ŇIn teachers they trustÓ on page 3].
Two steps forward, one step back
While the outsized weighting for student progress is a welcome step forward, problems and dilemmas remain.
First, the progress reports are based solely on the state ELA and math tests in the elementary and middle schools, so the achievement measures are only as good as the tests themselves. Then of course, students, and schools, get credit only for reading and math performance. Progress or excellence in science or social studies or music or art or physical education or language doesn't count.
Early childhood achievement doesn't count either. Children in pre-K through 2nd grade aren't tested, and 3rd grade is used as a base year. Progress can only be measured starting in 4th grade. So for 3rd-graders, only absolute performance counts.
Another problem involves the qualitative measures. Most educators would weight school environment factors a lot more than 15 percent. Creating a classroom environment conducive to learning and a school climate that supports teaching are the hallmarks of educational greatness.
Using the data
Expect great consternation when the schools receive their progress reports. ItŐs not easy to understand how the grades were awarded. People will question whether they are fair, and whether good aspects of schools were overlooked.
It took two days of workshops for UFT representatives to fully understand the reports. The formula for determining the final grade a school receives looks like something scrawled across the blackboards of a mad cartoon professor.
School progress in each area is not only recorded but measured against both a citywide average and, more heavily, against the average of a peer group of schools with similar demographics and performance. That is done for each of three or four domains in each of ELA and math in each of three sections. Then the schools are ranked on a scale with cut scores assigned by percentages for As, Bs, Cs, Ds and Fs. Math teachers: Be forewarned. You may be called on to answer lots of questions.
And as one UFT rep noted, explaining the calculation to parents could be nearly impossible.
Which creates a two-sided dilemma. If they cannot be easily understood by parents and the public, then do progress reports really serve as a fair, transparent evaluation of a school? But the flip side of that question is equally disturbing: If a school is simply given a grade based on its percentages of students meeting standards, does that fairly measure the learning that took place?
That is exactly the problem of NCLB. It fails to credit many schools that are doing well with hard populations.
Perhaps the best that can be said of the new progress measures is that, well, they represent progress. There is a long way to go before teachers, parents and administrators can agree on and trust a complex, multi-layered evaluation of schools. But it is unlikely weŐll return to the earlier pass-fail assessments. The new assessment train has already left the station.
