Insight

Evaluating teachers: Beware of formulas

Jul 2, 2009 3:04 PM

Something you learn from watching teachers work: There’s more than one way to skin a cat. Effective teaching can be, and often is, nothing like the instruction book. Some effective teachers are tough and stern; some are warm and sweet. Some lecture — a no-no in K-12 pedagogy — yet the students are all but eating out of their hands. Some doggedly follow the curriculum guides and the very predictability seems to calm students and support their learning. Others mostly ignore the written curriculum (another no-no) and some of the most difficult students seem to thrill to the roller-coaster that is an inventive, personalized lesson plan in a seemingly seat-of-the-pants curriculum.

So what’s good teaching? Knowing that they practice both art and science, how do we evaluate teachers and help them improve? Experienced observers sometimes say they know it when they see it. But observers have their biases. Others say effective teaching may be better measured by how well students demonstrate learning on standardized tests. But tests have their biases, too. Measuring learning is not like applying a formula.

The prospect of trying to quantify teacher quality — and deliver the findings to a high-stakes marketplace — seems seriously impossible. This may explain why the process is in its infancy in U.S. public education. According to a new report, “The Widget Effect,” by The New Teacher Project, “most teacher evaluations are based on two or fewer classroom observations totaling 76 minutes or less.” New teachers get exactly six minutes more observation time on average than tenured teachers. And 73 percent of the 15,000 teachers, probationary or tenured, in the report’s survey said their most recent evaluation did not identify any areas for improvement or development.

Most evaluations don’t mean much anyway, to judge by the report’s findings. Many are (like New York’s) binary — just an S or U. In five school districts examined by TNTP using a binary system, more than 99 percent of teachers received S’s for all the years studied. In another five districts that use multiple rating systems, such as unsatisfactory, satisfactory, proficient and excellent, more than 70 percent of teachers were rated excellent.

When they graded themselves on a scale of 1 to 10, 43 percent of teachers, novice and tenured, gave their instructional performance a 9 or 10. “These teachers are not irrationally inflating their estimate of their teaching performance,” the report concludes, “they are simply responding to an environment in which all are assumed to be superior performers.”

But this doesn’t mean great teaching is taking place in 99 percent of classrooms. Asked to evaluate their colleagues, 43 percent of teachers in the 12 districts that TNTP surveyed said there was a tenured teacher in their school who should be dismissed for poor instructional performance, but hadn’t been.

Given that teacher evaluation is hard, subjective, upsetting and necessary, what should be done to make it work right?

When the New York City Department of Education first grappled with this issue in 2007, it hired a team of lawyers and charged them with figuring out ways of removing tenured teachers. Things could only get better from there.

And it appears they did. The New York City-based New Teacher Project brings a far more sophisticated understanding to effectiveness (and one of the co-authors of the report is the DOE’s former top lawyer, Dan Weisberg). The UFT has often been critical of TNTP. But this time, the project sought out teacher unions and education experts to advise it. The report credits the advice and participation of 50 district and state education officials and 25 teacher union representatives, including teachers and union leaders in Denver, Akron, Cincinnati, Columbus and Toledo. And when the Toledo local disputed some of the report’s the findings, TNTP immediately offered to sit down with union leaders and review the data until they got it right.

The notion of teacher evaluation as identifying and persecuting “bad” teachers is notably absent from this report. Instead, it criticizes “the widget effect,” viewing teachers as interchangeable factory parts. “In general, our schools are indifferent to instructional effectiveness — except when it comes time to remove a teacher,” it charges. “Effective teaching must be recognized; ineffective teaching must be addressed … The core purpose of evaluation must be maximizing teacher growth and effectiveness, not just documenting poor performance as a prelude to dismissal.”

The report’s recommendations — that administrators get far more training, and pay far more attention to evaluations, done for the purpose of improvement, not dismissal — are a sea change in the tone and tenor of the national discussion on teacher quality. In this way, “The Widget Effect” reflects the progress in education debates ushered in with a new president and education administration, and with the growing influence of the American Federation of Teachers in the teacher quality discussion.

For example, there is the strong warning that teacher “value-added” scores, derived from student test data (as they are in New York City’s Teacher Data Initiative), should not be used as a substitute for comprehensive evaluation and observation. (The UFT believes that in its current state, value-added scores have no place in evaluations, and we supported the state law that prohibits their use for awarding tenure.) The report’s emphasis on professional development, based on observed strengths and weaknesses of individual teachers, is light years ahead of the view that teachers should know everything on Day One or be dismissed.

However, the recommendations are broad and ambitious. Teachers will resist them, and with good reason, given top-down school cultures where principals call all the shots. They propose creating a world that does not exist in most schools. They envision extended evaluations with individualized professional development for all teachers. And they suggest using new evaluation systems to make decisions in teacher assignments, compensation and dismissal.

It will be nearly impossible to do this without giving teachers more of a role in evaluations, something the report skirts around. And evaluating teachers based on student performance requires far better tests than most states now use (not to mention fair and accurate calculations of teachers’ effect on student scores).

In Toledo and Cincinnati, where peer review systems are quite advanced, it’s become evident that teacher colleagues, not just administrators and supervisors, play an essential role in helping novice teachers improve and assuring that tenured teachers succeed (or leave). In those districts, teachers and administrators have learned that good evaluations are “formative and supportive,” in the words of Julie Sellers, president of the Cincinnati Federation of Teachers. Test data is used to improve practice. That means it’s done with teachers, not to them.

Evaluating teachers is neither simple nor easy, nor should it be. If we want the best for each student, we must recognize teachers’ individuality, and be careful not to reduce effectiveness to a formula. That requires a radical change in public school culture — and that, sadly, is not yet on the nation’s education agenda.