Judge in NY State Throws Out ‘Value-Added Model’ Ratings

I am pleased that in an important, precedent-setting case, a judge in New York State has ruled that using Value-Added measurements to judge the effectiveness of teachers is ‘arbitrary’ and ‘capricious’.

The case involved teacher Sheri Lederman, and was argued by her husband.

“New York Supreme Court Judge Roger McDonough said in his decision that he could not rule beyond the individual case of fourth-grade teacher Sheri G. Lederman because regulations around the evaluation system have been changed, but he said she had proved that the controversial method that King developed and administered in New York had provided her with an unfair evaluation. It is thought to be the first time a judge has made such a decision in a teacher evaluation case.”

In case you were unaware of it, VAM is a statistical black box used to predict how a hypothetical student is supposed to score on a Big Standardized Test one year based on the scores of every other student that year and in previous years. Any deviation (up or down) of that score is attributed to the teacher.

Gary Rubinstein and I have looked into how stable those VAM scores are in New York City, where we had actual scores to work with (leaked by the NYTimes and other newspapers). We found that they were inconsistent and unstable in the extreme! When you graph one year’s score versus next year’s score, we found that there was essentially no correlation at all, meaning that a teacher who is assigned the exact same grade level, in the same school, with very similar  students, can score high one year, low the next, and middling the third, or any combination of those. Very, very few teachers got scores that were consistent from year to year. Even teachers who taught two or more grade levels of the same subject (say, 7th and 8th grade math) had no consistency from one subject to the next. See my blog  (not all on NY City) herehere, here,  here, herehere, here, here,  herehere, and here. See Gary R’s six part series on his blog here, here, here, here, here, and here. As well as a less technical explanation here.

Mercedes Schneider has done similar research on teachers’ VAM scores in Louisiana and came up with the same sorts of results that Rubinstein and I did.

Which led all three of us to conclude that the entire VAM machinery was invalid.

And which is why the case of Ms. Lederman is so important. Similar cases have been filed in numerous states, but this is apparently the first one where a judgement has been reached.

(Also read this. and this.)

A New-ish Study Showing Problems with Value-Added Measurements

I haven’t read this yet, but it looks useful:

http://www.ets.org/Media/Research/pdf/PICANG14.pdf

Published in: on December 12, 2013 at 10:02 am  Leave a Comment  
Tags: , , ,

An Article By A NYC Charter School Teacher for “PolicyMic”

I thought this was worth reprinting:

==========================

Hi all,

I submitted this article about my experiences re: standardized testing and test-based evaluations in NYC schools to PolicyMic this afternoon.  I hope that it resonates with many of you!  Please feel free to read, comment, and forward as you wish.

And, in honor of Mr. Wellstone (July 21, 1944 – October 25, 2002), who reminded us that laborers and teachers are one in the same, “Stand up. Keep fighting.”

Thanks!
Allison LaFave

http://www.policymic.com/articles/17490/romney-loves-teachers-what-teacher-evaluations-and-tests-mean-for-american-teachers

Romney Loves Teachers: What Teacher Evaluations and Tests Really Mean for American Teachers

During Monday’s final presidential debate, Bob Schieffer spurred a collective American chuckle when he cut off Romney’s long-winded brown-nosing with the knee-slapper, “I think we all love teachers…”

I’d love to believe Mr. Schieffer, but as someone who hails from a family of public school teachers and spent last year teaching third grade in a New York City charter school, I have to say, “Bob. You’re adorable. But America’s teachers haven’t felt loved in quite some time.”

Last spring, my principal corralled our school’s third grade teaching team around a kidney-bean shaped table and apologetically explained that we needed to sign forms acknowledging the weight of our students’ test scores on our end-of-year evaluations. Ultimately, our students’ math and ELA scores would comprise as much as 40% of our annual rating.

Now, I don’t know a single educator who outright opposes the idea of fair evaluations and/or some level of teacher accountability. But as I sat quietly in that little red plastic chair, a voice in me cried:

“You want to evaluate me? Great. No problem.

“But let’s also evaluate the misaligned (or nonexistent) curriculum I was given to plan for my classes.”

“Let’s evaluate the number of chairs huddled around single desks, because there are more students in the room than there were last year, and the copy machine, the one that never works.

“Let’s evaluate the number of students with IEPs that aren’t being adequately serviced, and the number of English Language Learner students sitting voiceless in the back of the room, because they have yet to be admitted into nonexistent ELL classes.

“Let’s evaluate the employers who are smugly underpaying/underemploying my students’ parents or guardians, forcing them to work multiple jobs, likely without ever securing benefits for themselves or for their families. Or the number of students who have lost parents or loved ones due to gang violence, substance abuse, or the labyrinth that is our failing criminal justice system. Or the number of my students who didn’t eat dinner last night.

“Let’s evaluate how many hours of sleep I got last night, because I was not afforded adequate prep time during my 10 or 11 hour day in the building, or how many times I’ve skipped out on doctor’s appointments and family events to be here for my students.

“And, finally, let’s evaluate my motivations for being here — because it sure as hell isn’t for the money.”

Last week, Deborah Kenny wrote an op-ed piece decrying the heavy influence of test scores on teacher evaluations. Kenny rightfully claimed that the practice “undermines principals and is demeaning to teachers” and leaves little room for innovative teaching and learning. She went on to say that test-based evaluations inhibit the “culture of trust” between principals and teachers and “discourage the smartest, most talented people from entering the profession.”

While I agree that test-based evaluations are inherently flawed (when was the last time our politicians, Democrats or Republicans, truly analyzed aPearson test?), I am baffled by Kenny’s ultimate argument. It seems that Kenny bashes test-based evaluations because … wait for it … they make it harder for her to fire teachers she doesn’t like – specifically a teacher whose students performed “exceptionally well” on the state exam.

Teachers aren’t statistics, but they also aren’t part of some school-wide homecoming court. Administrators shouldn’t cast votes for the teachers they like or dislike. They should work to support all teachers who act in the best interest of students.

Ms. Kenny also takes a not-so-subtle jab at teachers’ unions, attacking evil tenured teachers in America, who are clearly exploiting their glamorous roles as K-12 educators. However, unions don’t grant tenure; PRINCIPALS grant tenure. And, moreover, Ms. Kenny, like nearly all charter school administrators in America, likely prohibits her teachers from joining their local union.

As someone who has worked in a non-union school, I can tell Ms. Kenny what violates trust between teachers and administrators. Knowing that you can be fired for your personality.  Knowing that there is a fresh crop of well-intentioned, starry-eyed Teach for America kids who can take your place in the time it takes to make a phone call. Knowing that you will be scorned for using your allotted sick days and guilted into working through lunch, during prep time, and hours after the final school bell rings.

I encourage our presidential candidates (and all Americans) to listen to the voices of practicing teachers, who are so often talked about and around during national education debates.

Says Kelly G., a third grade teacher in Brooklyn:

“These teacher evaluations are complex. I honestly used to think that a teacher could indeed be evaluated and held accountable using test scores. And then I started teaching at school that didn’t allow me to do the kind of teaching I thought needed to be done in order to develop intelligent children. There’s nothing quite like having your teaching micromanaged and then being told it was your fault the kids didn’t achieve exemplary scores on the state exam.

“My kids are capable of so much already. Come in and look at their writing. Listen to their discussions. Watch them solve math problems. Their tests scores will not reflect their growth from the school year. A one shot assessment does not give a good picture of student achievement. Have you read those exams? Have you been in the room during testing? Test anxiety vomiting is a real thing in the third grade. Too bad they don’t evaluate me on sick child comforting and vomit clean up. I’m sure my scores on those evaluations would be proficient.”

In popular media, teachers are cast as heroes or villains. They are either lazy, money-grubbing, ne’er-do-wells or Jaime Escalante, the “teacher savior” of the acclaimed film Stand and Deliver.

The truth is, as in most professions, the majority of teachers lie somewhere in the middle of this spectrum. Such romanticized notions of teaching make great stories, but that’s just it; they are stories that too often exaggerate and obscure the truth. Jaime Escalante spent years preparing his students for the AP Calculus exam, not a few inspired semesters. Does that mean that he was an inadequate teacher during the years he spent honing his craft and teaching foundational math concepts to his students? How would Escalante have been rated under the New York City evaluation system?

In his research paper entitled “Effects of Inequality and Poverty vs. Teachers and Schooling on America’s Youth,” David C. Berliner (Regents’ Professor Emeritus in The Mary Lou Fulton Teachers College of Arizona State University) finds that “Outside-of-school factors are three times more powerful in affecting student achievement than are the inside-the-school factors.”

Consequently, he concludes, “The best way to improve America’s schools is through jobs that provide families living wages. Other programs…offer some help for students from poor families. But in the end, it is inequality in income and the poverty that accompanies such inequality that matters most for education.”

America’s education system is in crisis; of this, we can be sure. But let’s stop blaming the dentists for their patients’ cavities.

Follow

Get every new post delivered to your Inbox.

Join 461 other followers

%d bloggers like this: