Teacher Data Reports

The Department of Education in 2010 was preparing to turn over to the media the names and highly debatable data of 4th- through 8th-grade English language arts and math teachers. The Teacher Data Reports are based on the students’ standardized test scores, which themselves were found to be inflated and inaccurate.

The release of the names has been halted pending the outcome of a lawsuit filed by the UFT.

Despite a written promise to the UFT in 2008 that the reports “will not and should not be disclosed,” the DOE was readying the reports for five New York media outlets, which had requested the information under the Freedom of Information Law.

Aside from privacy concerns, the UFT fears that the reports are so inaccurate that they could badly mislead the public.

The reports use a new and complex statistical technique called value-added measurement to try to determine the impact of an individual teacher on a student’s test score. Value-added measures are intended to level the playing field for teachers of higher-needs and lower-needs students by measuring improvements rather than absolute proficiency, thus enabling comparison between teachers. But the technique still needs refinement and is the subject of ongoing national debate. As of now, the measure is “not ready for prime time,” according to researchers. The DOE itself acknowledges that the reports have super-wide margins of error.

The UFT has assisted researchers to refine the value-added methodology in hopes that it could someday become a useful instructional tool. It also agreed with the State Education Department last summer that value-added measures could be one part of teacher evaluation when and if they could be made reliable. Read more about he UFT's work to develop a new teacher evaluation system.

But the UFT considers the current reports to be flawed and unreliable for the following reasons:

1. The student test scores on which these reports are based have been discredited. With testing experts from Harvard University now finding the New York State exams have been scored differently every year and tested only a narrow band of knowledge, reports that are based on them cannot give good information.
2. The data reports are based solely on the test scores. Even if they were reliable, the test scores would be an inadequate and inappropriate measure of teacher effectiveness. Any meaningful measure of teacher effectiveness must rely on multiple measures.
3. The “value-added” measurement that the reports use keeps giving different results. Teachers’ outcomes vary widely from year to year, or classroom to classroom. The DOE reports give error margins on average of 54 percentage points for teachers with one year of data, because the statisticians just aren’t sure of their reliability.
4. “Garbage in, garbage out” is an old saying about computer calculations. It’s true for the data reports. Multiple instances have come to light of teachers being graded on students they never had or subjects they never taught. If the inputs aren’t right, the outcome can’t be either. Use this guide to check if your report has errors.

See UFT President Michael Mulgrew’s Oct. 20, 2010, email message to members >>