Top News Stories
How to read your teacher data report
Dec 11, 2008 3:08 PM
Starting in December, principals will give grades 4 through 8 ELA and math teachers their “Teacher Data Initiative” reports. The reports rate teachers on the changes in their students’ statewide test scores in an attempt to identify how much of that is attributable to the work of the individual teacher.
This “value-added” method is often considered fairer than judging teachers by unadjusted test scores. It looks at students’ progress from their individual starting points and tries to statistically account for factors outside a teacher’s control, such as demographics, attendance and class size.
Some teachers will find the reports useful, others won’t. The Department of Education has invested heavily in this new measurement system, but the big test is still ahead: will this actually help teachers improve their teaching?
At the UFT’s insistence, the reports cannot be used in teacher evaluations, and they cannot be made public [see “Data reports can’t be used to evaluate teachers”]. Only your principal and other school administrators may see the reports; they may not be shared with colleagues without your permission.
The summary tables (below) from the front page of the ELA report of a fictitious 5th-grade teacher, “Mary Travis,” are reproduced and explained. (The inside pages of the report give further details and numerical scores. The back page has explanations to help interpret the data.) High school teacher reports are in the works but are expected to look quite different.
Calculating ‘progress’
The measurement starts with the previous-year scores of each teacher’s students and then predicts this year’s expected score, factoring in dozens of factors known to affect performance, like poverty, summer school attendance, special education and language status. How much progress did the teacher make with his or her students? How about with specific subgroups of those students? And how does that compare with other teachers of the same grade and subject in the city? How about compared with “peer” teachers, those who have similar years of experience and teach similar students? If the data are available, the same is done for two previous years.
Now what?
Once teachers have this report, they should compare it with other information they have about their students and their teaching. Does the report affirm or contradict other evidence? No one piece can give a full and true picture of a teacher’s skills.
The last step is for teachers to think about what they learned to help them improve their teaching. For example, on which group(s) of students does Ms. Travis need to focus her improvement efforts? She might want to reflect on what she does with other groups with whom she is more successful to see if those strategies would work with her less successful students. Or she might want to think about what she is doing this year as compared to last year. Finally, she may want to ask colleagues who have success with a particular subgroup if she can visit their classrooms or discuss strategies with them.
In the broadest comparison, Ms. Travis is at the 58th percentile of all 5th-grade ELA teachers, meaning she does as well or better than 58 percent of 5th-grade ELA teachers citywide in improving her students’ test scores. However, her results should be read as “give or take” 19 points. That 58 percent is the best guess in an estimated performance range between the 39th and 77th percentiles, shown by the horizontal bar on the right.
Ms. Travis’ performance has gone up this year from a 49th percentile average over the last three years. Again, though, that 49 score is really plus or minus about 13 points. While her result is most likely to fall toward the center of the range, near the little carat, it could reasonably fall anywhere within that range.
Here Ms. Travis is compared with peer teachers, meaning other 5th-grade ELA teachers at her experience level, teaching students who are expected to make similar gains. Again, her 65th percentile result must be read as “give or take” 19 points, in a wide range of 46 percent to 84 percent (from slightly below average to considerably above).
Compared with her peer teachers, Ms. Travis does very well with low performers and special education students (green zone) and about the same as the middle 60 percent of teachers with the groups listed under the yellow box. However, Ms. Travis does not do as well with high-performing students (orange zone). With them, she scores among the lowest 20 percent. But note the asterisk next to that result. It indicates that the range is so large (because there is only a small amount of data) that the results should be interpreted with caution. Ranges like this are why these reports cannot be used to evaluate a teacher’s true effectiveness.
Go here to read the joint Klein/Weingarten letter in the Oct. 1 Principal’s Weekly on the Teacher Data Initiative.
