Principal Rightly Slams Coporate Junk that Masquerades as Standardized Testing
The Valerie Strauss of the “Answer Sheet” has hauled in another good op-ed article this week. She found a principal who had the intelligence and integrity and good writing skills to eviscerate most of the reasons given for the current DEformista educational leadership these days. I hope to read more from this person, whose name is Sharon Emick Fougner, principal of Elizabeth Mellick Baker School in Great Neck, N.Y.
A quote from Ms. Fougner that particularly caught my eye:
What is even more detrimental is that neither these children — nor their parents or teachers — will ever have access to their test booklets in order to understand how or why the child arrived at an incorrect answer. No benefit is extended to the child from all of these hours of testing if there is no thoughtful, comprehensive feedback. Likewise, I am unable to provide you and your department with clarification and examples regarding my initial list of concerns, as I am not permitted to speak about the content of the exams, or retain a test booklet for commentary. I find it disingenuous that you want teachers and principals to receive feedback, but want none yourself. It would seem that those of us who have spent our lives doing this work would have much insight to offer you.”
Among other things, the crisp writing here explains in detail why this principal’s criticisms of the overall poor quality of the test itself, seemed so … vague. The reason is that IF SHE HAD TRIED TO BE MORE SPECIFIC IN ANY WAY by keeping a copy of the booklet to review, or merely making a copy of or notes about any test item, or even to discussing any of the items orally or in writing, as in this op-ed, she could end her career.
Late in my teaching career, I did something I figured was generally frowned on, but I thought was a good and principled thing to do.
What was my crime?
I carefully read the math questions and classified them according to the mandated city curriculum of the day by making pencil checkmarks on a copy of said curriculum.
Oh! The horrors! Arrest that man! Take away his pension! Publicly embarrass him!
One day my adorable class of 7 or 8th grade black, white, asian, and latino public school with mostly fairly well-involved parents were taking one of these standardized NCLB-mandated annual tests. This is a process that drags on for days of almost non-stop inactivity. My students — even the very slowest among them — generally finished each section in about one-quarter to one-half the time allotted for each section. During the remainder of the time (which could be 5 to 35 minutes depending on the section, they had to sit there, bored, doing — NOTHING.
No, you may not go on to the next section.
No, you may not talk, even to yourself.
No one may turn around or stretch or make faces.
They may not write.
They may not draw.
They they may not read.
They certainly can’t call up their friends on their personal electronics (a double-edged sword that I only caught the beginnings of, thank my lucky asteroid!)
They may not even put their heads down and go to sleep.
Teachers are instructed to go around to students, reminding them to re-read every single question in the current section of the days-long test, to re-check their thinking and their work, and change answers or not as they see fit, but in any case, to re-check their work.
And if your homeroom is good and compliant you have to go around quietly in the last portion of the time allotted quietly whispering in your students’ ears:
“No you cannot go to the bathroom unless it’s an emergency, and if it really is, you’ll have to walk with the other proctor to the bathroom.”
“No, sorry, you cannot take out a book and read it. Just sit there quietly and space out for a few minutes by zoning into space, and then look at your test questions again. Thank you. Yes, I know, they certainly do give us too much time, and yes, honey, this sure is boring to have to do for four to eight weeks every single school year. Teachers mostly don’t like it, either, sweetie, we know just about everybody has finished long ago, even the kids who get extra time on normal classroom activities. Nothing I can do. ”
Today, if I were still teaching, I might also quietly suggest, “Honey, if you don’t like this, and you know a lot of your teachers think this is a scary waste of time, and I myself agree, your parents go to PTA meetings, right? Have you discussed this with your parents? Are they on any official or unofficial PTA policy boards?”
so many folks love to have multiple-choice tests: checking them takes no time a-tall. It’s the making up of the questions that’s hard, and involves a lot of trial and error; and during that process of field-testing a multiple-choice item, it needs to be kept secret, since it’s so difficult to come up with good ones. Or else the testing companies do what all big companies do: hire cheap labor and try to automate the process. So they hire a bunch of people who are literate but desperate for work, and they pay them so many pennies or dollars per test item, most likely. Obviously, the more test items these folks who are now “associates” or “independent contractors” or “consultants” (but we don’t use legal terms like ’employees’ for solid $$ reasons, and it’s no longer polite to call them ‘low-paid unorganized hacks for hire’). the more money these poor souls earn, but it’s not much. The really big bucks go to shareholders and directors of groups like Kaplan, CTB-McGraw-Hill, Pearson, and so on, who have swallowed up nearly the entire educational publishing industry in this country. (Something like what happened to bookstores.)
In any case, that’s what was behind the Sleeveless Pineapple story of a few days ago. The article Valerie Strauss has here shows some other reasons why the current approach is all wrong.
To sum up the problem:
(1) First of all, there is no such thing as a perfect question, be it multiple-choice or essay. I’ve seen and written some good ones, and I’ve seen and written some bad ones. None of them ever do more than give you a peek at what the child knows and understands.
(2) There are quite a few teachers who for years have been coming up with way better questions for assessing student understanding than what is in most standardized publisher-driven stuff. But the current edu-DEform racket ignores anything coming from a teacher who is in the classroom.
(3) All of the actual questions on these standardized tests are kept secret, so that students and teachers and parents never get the opportunity to see what exactly Johnny or Susie messed up on, or whether the questions the students missed were merely poorly written.
(4) Panels that I or my former colleagues sat on, often rejected or voted to strongly reword most of the proposed commercially-created NCLB questions we were asked to review and critique. But even though we threw out most of what they wrote, they managed to come up with even more crappy questions that get used year after year. Guess they found another panel that wasn’t as picky as us? Or they stopped having teachers review them? These secret NCLB tests are still crappy. These writers must never have taught this course in the first place, or they are simply lazy, or idiots. I can only guess which option it is — and none of them are very good now, are they?
(5) Released questions from DC’s Comprehensive Assessment Program do not inspire any confidence either; see my previous columns in this blog. Do a search for “released questions” or “DC-CAS”.
(6) The coverage of the curriculum is extremely spotty. By my own count, and according to what the local testing honchos state, there are in math often dozens of questions on a single topic in the curriculum (out of a list of nearly a hundred topics to be ‘covered’ during the year. As a partial consequence, many, many topics listed in the yearly curriculum are not addressed with even a single question.
(7) As the writer of this op-ed states, many questions are ambiguous at best. Based on the released DC-CAS items, I conclude that this defect happens here in DC as well. Assuming that this suburban Long Island principal writes is true, this sort of stupidity is now even getting worse than it was when I was teaching. This really does sound like child abuse.
(8) Forcing kids to be motionless for hour after hour, day after day, is not really what we want our kids to be doing with their time. Is it? They don’t make kids in progressive and expensive private schools do this sort of thing. Why should the public school kids have to do it, then?
(9) The local DC not-quite-NCLB pre-tests, which are manufactured by a competing or allied company and which are supposed to be used by teachers to create even more data and more instruction, are of even worse quality here in DC than the actual NCLB end of year tests. They create entirely erroneous data.
(10) Garbage In, Garbage Out. Much of the incoming data is useless, yet it’s supposed to be used to ‘guide instruction.’ It’s analogous to saying “If those pigs over there are flying under their own power, then I have $20 billion in gold ingots in my closet.’
Back to my little story.
One day, one of my homeroom students was out sick on several days of April testing week. (My homeroom class at this school this year generally had excellent attendance.) While my students were quietly working away on the test and being model citizens, I decided to take the question booklet (no, not the answer sheet! Why would I do that? I was genuinely interested in how well the test matched what I had been teaching. That’s all. I was not about to answer his/her questions for him/her!)
I took a well-worn copy of our district-wide mandated curriculum, which I had just spent a summer analyzing, and I read every single math item on the NCLB test, one by one. After reading each question, I then put a check mark next to the curriculum topic or standard or whatever they are calling it these days that best matched the question — in my opinion. And I think my opinion was pretty good since, as I said, I’d been analyzing it all one summer.
I wrote nothing else.
In case you are wondering, I only did this inspection this at times when my students were engrossed in actual test taking, generally from about 10% of the way in to the time of the session, up to about 55% of the time allotted. I would then put it back into the carefully-alphabetized and sorted testing material during the last 45% of of each section. (For me, it doesn’t take long. Repeat: I didn’t write down answers; I just noted what skills or standards were being tested for each question.
That year, testing was quite easy, because my students were wonderful. Much of the time, it was quite easy for one proctor to watch the class while the other one might go to the rest room or run off a few photocopies and come back.
What I found shocked me.
I kept the list with the little tickmarks with me when I retired (I hope), and later turned that data into a spreadsheet, which I published a couple of years ago now on this blog, IIRC. The curriculum standard that deals with computation was hit over and over and over again with question after question. Now, many folks think that that’s all math is: computation. That’s actually not true It’s an important part, but there is much more as well.
I found that more than half the curriculum wasn’t addressed by any question whatsoeve. Many topics in statistics, data analysis, probability, measurement, geometry, and graphing, that were listed for teachers to teach, weren’t tested.
It’s good to see someone writing so eloquently and clearly about this. I wish more teachers and administrators in DC would speak up — but I’m not surprised that they are all in fear of losing their jobs. As hundreds of them are about to do, starting today.
Thanks, Ms. Fougner, and my apologies for running on.