Speakout Columns
Brownie the Cow — and other unanswerable questions
Mar 1, 2007 3:17 PM
In January 2006, with other 4th-grade teachers across the city, I administered the three-day ELA test to my class. I was hopeful — for my students, that they would be able to utilize their comprehension and writing effectively, and for myself, that I could use the test results to improve and focus my teaching.
On the second day, I administered the listening section of the test in which a passage is read aloud by the teacher while students take notes. Students then answer several short questions and a long essay question related to the passage. Before the test started I only had a moment to glance at the passage I was to read — “Why the Rooster Crows at Dawn” — but it immediately struck me as over-simple and not very well written.
After reading the passage aloud twice, as instructed, I began circulating around the room, glancing at my students’ tests as they worked. I read the essay question for the first time. My heart sank. The test had a typo in it that distorted the entire focus of the question.
The story was almost completely about a rooster and his motivations and actions. Yet the essay question asked how Brownie the Cow, a minor character whose thoughts and motivations were not expressed at all, had “changed” during the course of the story. I was positive it was a typo, that’s how baffling the question appeared.
The next day, I was shocked to find that the reading response essay question of the test was equally confusing, if not more so. (The test questions need to be seen to be believed — see for yourself at browniethecow.org.)
After the exam, my students felt confused and dejected. I had told them they were well-prepared for the test and yet the test questions had baffled them. Several told me they thought the test had been “a trick,” with answers that couldn’t possibly be figured out based on the texts provided. There were tears, too. “Now I can’t get into the middle school I wanted to go to,” one girl sobbed.
In the days following, I talked with my colleagues and did a lot of thinking. Testing was something I hadn’t given much thought to before. I felt guilty: I dedicate myself to my students’ learning and educational well-being but I had just given them a test that didn’t measure what they knew. As far as I could tell, this test was too substandard to measure anything.
I wrote to Chancellor Joel Klein to voice my concerns. He forwarded my letter to a testing official, who replied with a “tests are designed to measure standards” form letter. Another brush-off from Tweed.
Things were becoming clear. And the clearer they became, the angrier I got. There was no typo on the essay question; it was simply that the test was awful — confusing, poorly written, unfit for any students.
This test was created by the State Education Department and CTB/McGraw Hill, a testing company that has been cited dozens and dozens of times for flaws in both its test questions and grading of tests.
When the ELA results came out last fall (nine months after the kids took the test), only one in 17 New York City 4th-graders had scored a 4 — a drop of 60 percent. I found the ELA test online and shared it with some parents and other teachers at my school who were as shocked as I was by how poorly written and confusing it was. I also found that some of my friends who have Ph.D.s in literature were unable to come up with the “right answer” to the Brownie the Cow question.
A parent at my school called the SED and talked to two employees about the test. One, from the Office of Assessment, told him the state’s answer to the Brownie question: Brownie “starts out nice and becomes mean,” an answer the parents I was working with and I found to be a huge stretch, if not just plain wrong. She also shocked this parent by telling him “the smart kids and the analytical kids have problems with these questions. They drive themselves crazy looking for the right answer.” The point of the questions, she said, was just to “get students writing” and that the writing was evaluated by “how you express yourself,” not the content of what you say.
As a teacher, I was infuriated to hear this. My students were ready to answer an essay question well. Had I taught them to “just write,” to say a lot of nothing, I would have been horribly irresponsible. And yet that’s exactly what the state wanted them to do.
After this, it was no surprise when this state employee admitted that though teachers were supposed to be involved in developing the tests, “only a handful of teachers were involved.” I wondered if the state and CTB/McGraw Hill had even listened to those teachers. No teachers I know would greenlight such awful test questions.
Another very serious concern is how much is riding on the results of these bad tests. Some middle schools demand 3s or 4s as a basis for admission; but what happens when the test is confusing and unfair and the scores plummet? None of our kids’ futures should depend on a single test and the 4th-grade tests were never intended to be used for something like middle school admissions. That was Klein and Bloomberg’s call. Klein is misusing the results of a bad test and children’s futures are at risk because of it. And the thing is, he doesn’t need to use the tests for this purpose; there are many other, fairer ways to measure student performance for the purpose of admissions.
Moreover, teachers and principals are evaluated based on test results. We are being evaluated based on the results of a horrible test.
I didn’t plan to get involved in the fight over standardized testing. And I’m not convinced all standardized testing is bad. But when I saw how much standardized tests can affect both students and teachers, and how unfair a bad test is, I knew I needed to do something. Klein needs to stop misusing the tests and, with parents and other teachers, I will keep pushing until he does.
I urge all of you to start thinking about this problem. When you see a bad test, complain.
Write to Klein to protest that tests are being used for purposes for which they were never intended. Spread the word to parents. Discuss the issue at your chapter meetings. Support the UFT’s mission to find a position on high-stakes testing. This issue affects our students, our profession and our taxpayer dollars; be a part of the push for a change in testing. We can’t afford to stand by any longer.
