A very funny critique of what a Delware Edu-Political boss actually meant:
I thought this was worth reprinting:
I submitted this article about my experiences re: standardized testing and test-based evaluations in NYC schools to PolicyMic this afternoon. I hope that it resonates with many of you! Please feel free to read, comment, and forward as you wish.
And, in honor of Mr. Wellstone (July 21, 1944 – October 25, 2002), who reminded us that laborers and teachers are one in the same, “Stand up. Keep fighting.”
Romney Loves Teachers: What Teacher Evaluations and Tests Really Mean for American Teachers
During Monday’s final presidential debate, Bob Schieffer spurred a collective American chuckle when he cut off Romney’s long-winded brown-nosing with the knee-slapper, “I think we all love teachers…”
I’d love to believe Mr. Schieffer, but as someone who hails from a family of public school teachers and spent last year teaching third grade in a New York City charter school, I have to say, “Bob. You’re adorable. But America’s teachers haven’t felt loved in quite some time.”
Last spring, my principal corralled our school’s third grade teaching team around a kidney-bean shaped table and apologetically explained that we needed to sign forms acknowledging the weight of our students’ test scores on our end-of-year evaluations. Ultimately, our students’ math and ELA scores would comprise as much as 40% of our annual rating.
Now, I don’t know a single educator who outright opposes the idea of fair evaluations and/or some level of teacher accountability. But as I sat quietly in that little red plastic chair, a voice in me cried:
“You want to evaluate me? Great. No problem.
“But let’s also evaluate the misaligned (or nonexistent) curriculum I was given to plan for my classes.”
“Let’s evaluate the number of chairs huddled around single desks, because there are more students in the room than there were last year, and the copy machine, the one that never works.
“Let’s evaluate the number of students with IEPs that aren’t being adequately serviced, and the number of English Language Learner students sitting voiceless in the back of the room, because they have yet to be admitted into nonexistent ELL classes.
“Let’s evaluate the employers who are smugly underpaying/underemploying my students’ parents or guardians, forcing them to work multiple jobs, likely without ever securing benefits for themselves or for their families. Or the number of students who have lost parents or loved ones due to gang violence, substance abuse, or the labyrinth that is our failing criminal justice system. Or the number of my students who didn’t eat dinner last night.
“Let’s evaluate how many hours of sleep I got last night, because I was not afforded adequate prep time during my 10 or 11 hour day in the building, or how many times I’ve skipped out on doctor’s appointments and family events to be here for my students.
“And, finally, let’s evaluate my motivations for being here — because it sure as hell isn’t for the money.”
Last week, Deborah Kenny wrote an op-ed piece decrying the heavy influence of test scores on teacher evaluations. Kenny rightfully claimed that the practice “undermines principals and is demeaning to teachers” and leaves little room for innovative teaching and learning. She went on to say that test-based evaluations inhibit the “culture of trust” between principals and teachers and “discourage the smartest, most talented people from entering the profession.”
While I agree that test-based evaluations are inherently flawed (when was the last time our politicians, Democrats or Republicans, truly analyzed aPearson test?), I am baffled by Kenny’s ultimate argument. It seems that Kenny bashes test-based evaluations because … wait for it … they make it harder for her to fire teachers she doesn’t like – specifically a teacher whose students performed “exceptionally well” on the state exam.
Teachers aren’t statistics, but they also aren’t part of some school-wide homecoming court. Administrators shouldn’t cast votes for the teachers they like or dislike. They should work to support all teachers who act in the best interest of students.
Ms. Kenny also takes a not-so-subtle jab at teachers’ unions, attacking evil tenured teachers in America, who are clearly exploiting their glamorous roles as K-12 educators. However, unions don’t grant tenure; PRINCIPALS grant tenure. And, moreover, Ms. Kenny, like nearly all charter school administrators in America, likely prohibits her teachers from joining their local union.
As someone who has worked in a non-union school, I can tell Ms. Kenny what violates trust between teachers and administrators. Knowing that you can be fired for your personality. Knowing that there is a fresh crop of well-intentioned, starry-eyed Teach for America kids who can take your place in the time it takes to make a phone call. Knowing that you will be scorned for using your allotted sick days and guilted into working through lunch, during prep time, and hours after the final school bell rings.
I encourage our presidential candidates (and all Americans) to listen to the voices of practicing teachers, who are so often talked about and around during national education debates.
Says Kelly G., a third grade teacher in Brooklyn:
“These teacher evaluations are complex. I honestly used to think that a teacher could indeed be evaluated and held accountable using test scores. And then I started teaching at school that didn’t allow me to do the kind of teaching I thought needed to be done in order to develop intelligent children. There’s nothing quite like having your teaching micromanaged and then being told it was your fault the kids didn’t achieve exemplary scores on the state exam.
“My kids are capable of so much already. Come in and look at their writing. Listen to their discussions. Watch them solve math problems. Their tests scores will not reflect their growth from the school year. A one shot assessment does not give a good picture of student achievement. Have you read those exams? Have you been in the room during testing? Test anxiety vomiting is a real thing in the third grade. Too bad they don’t evaluate me on sick child comforting and vomit clean up. I’m sure my scores on those evaluations would be proficient.”
In popular media, teachers are cast as heroes or villains. They are either lazy, money-grubbing, ne’er-do-wells or Jaime Escalante, the “teacher savior” of the acclaimed film Stand and Deliver.
The truth is, as in most professions, the majority of teachers lie somewhere in the middle of this spectrum. Such romanticized notions of teaching make great stories, but that’s just it; they are stories that too often exaggerate and obscure the truth. Jaime Escalante spent years preparing his students for the AP Calculus exam, not a few inspired semesters. Does that mean that he was an inadequate teacher during the years he spent honing his craft and teaching foundational math concepts to his students? How would Escalante have been rated under the New York City evaluation system?
In his research paper entitled “Effects of Inequality and Poverty vs. Teachers and Schooling on America’s Youth,” David C. Berliner (Regents’ Professor Emeritus in The Mary Lou Fulton Teachers College of Arizona State University) finds that “Outside-of-school factors are three times more powerful in affecting student achievement than are the inside-the-school factors.”
Consequently, he concludes, “The best way to improve America’s schools is through jobs that provide families living wages. Other programs…offer some help for students from poor families. But in the end, it is inequality in income and the poverty that accompanies such inequality that matters most for education.”
America’s education system is in crisis; of this, we can be sure. But let’s stop blaming the dentists for their patients’ cavities.
Anybody who is interested in promoting the welfare of the 99% of the population who don’t have offshore bank accounts or manage hedge funds should be ecstatic over the apparent success of the members of the Chicago Teachers Union in beating back most of Rahm Emanuel’s educational DEform policies.
A lot has been written about this strike. I am thrilled that over 90% of the teachers voted in favor of the strike, that they stuck together, and that they “stuck it” to arrogant Rahm.
Let us remember that EVERYWHERE that the billionaires’ educational DEforms have been tried, they have failed utterly. I salute the Chicago rank-and-file who stood up.
Standing Up to Corporate School Agenda, Chicago Teachers Greenlight Strike
Vaulting the High Bar
It was suggested to me that the fact that very few teachers remained in the 90th percentile for two years in a row in NYC’s value-added madness (my description, not his) is simply yet another case of regression to the mean. That’s a phenomenon where very tall parents tend to have kids that are a bit shorter than they are, and very short parents have children that are taller than them.
Perhaps. But we definitely from ordinary observation that that height, hair color and skin color (but not tattoos or most illnesses) are rather inheritable: tall parents tend to have kids that are taller than most, and short parents have kids that are shorter than most, and so on.
But when the data is a blob showing almost no correlation at all, then regression to the mean doesn’t really mean the same thing. I mean, it starts to become just random variation. Yaknow whatI mean?
OK, let’s look at NYC again.
I just figured out how to get Excel to count some stuff for me in a neat and efficient manner. It counted for me all of the NYC public school teachers who were at or above the 80th percentile in the value-added measurement scheme they’[ve been using there for sy 0506; (That would be considered excellent.) I also had it check to see whether they were also in the 80th percentile during SY 0708, two years later. Or not.
If we trust my programming of Excel, there were exactly 161 such teachers who were in the 80th percentile rank or higher during both years.
But I also had it count how many teachers “dropped”, so to speak. I found there were 545 who were below the 80th percentile the second measured year. Oh, well, that’s nothing – that’s just regression to the mean, you say.
Well, what about those who go BELOW the mean the second year? That’s more than just regression to the mean. After all, the children of congolese pygmies do not get tall enough to play for the NBA. It would take a lot of intermarriage for several generations for that to happen (sorry about that, Bugsy Malone).
In New York city, I found that 146 teachers who were high-flyers in 2005-6 (at or above the 80th percentile) were distincly sub-par, scoring at or below the 50th percentile, two years later.
That’s close to the number of high flyers, nand is about 1/3 of those who “dropped.”
This ‘value added’ stuff is worthless. It has no real predictive value. It doesn’t tell us anything we really want to know, even on its own terms. Plus, it’s measuring the wrong things — but that’s the subject for many more columns to come, and not just by me.
Someone who professes to understand Value-Added scores better than me claims that my graphs for NYC are meaningless because the scores for 2007 were inflated; he claimed that the overall year-to-year and year-to-career value-added correlation coefficients are much higher than what I found — thus, VA is really useful, just not my particular graphs..
Taking this objection seriously, I decided to leave out SY 0607, and compare SY 0506 to SY 0708. Same exact teachers, same exact subjects and grade levels, same exact schools, obviously different (but quite similar) kids.
Here is the scatterplot of what I found. Again, I asked Excel to calculate a line of best fit, and it drew it. Notice that the r-squared correlation value is about 0.05 — seriously LOW. Notice also that this scatterplot is basically a blob again, again a classic example of one variable showing very little correlation with another. (West Virginia’s map has a much more defined shape!) In any case, there are lots (hundreds? thousands?) of teachers with positive VA scores in the first year and negative VA scores the third year, and vice-versa. Only an easily countable handful of teachers have scores of +0.2 or better both years, or worse than -0.2 both years. Out of all of the thousands of teachers. And I bet those are all accidents as well.
So, in other words, I find, as did Gary Rubenstein, that there is extremely little correlation between two things that should be, you would think, very close to a perfect 1.00 correlation. (In the real world, of course, you almost never get a 1.00 correlation between any real entities or quantities. However, when you are talking about the scores of teachers who have been teaching IN THE SAME SCHOOL, THE SAME SUBJECT, THE SAME GRADE LEVEL for three straight years, then you would think that their performances would be rather similar all three years. If anything, they would normally get better unless they had suffered some sort of physically or mentally debilitating injury or illness (often from old age and the incredible amount of stress). In particular, a lot of teachers will admit to you that they absolutely sucked at teaching during their first year, but that they then figured out a lot of those errors and tried not to make the same ones the next year, so they really improved, or else they quit. But these folks didn’t quit. These are at the very least three-year veterans, which in DC would make them eligibility for department or grade level chair at their school as a result of seniority alone, since so many of the older teachers have quit or retired, and the turnover and attrition over the last few years among the newest hires in our school system is probably unprecedented in the history of education. (Perhaps not, but it’s a subject I’d like to pursue.)
Finally, while I admit that I exaggerated a bit (for effect) when I said that the shapes of these graphs, and the very low computed values for the r-squared coefficient of linear correlation, made value-added about as predictive as numerology. I thought about that particular exaggeration and wondered how serious it was. So, even though I have participated in a fairly large number of courses on calculating probabilities and distribution, it’s always a bit fraught with error: Have we counted all of the possibilities? Have we left any out? Have we double-counted any of them? Is there a much better, faster, or less error-prone method hidden right around the corner?
To make a long story short: the Monte Carlo method is a great way of deciding, say, how likely something is to happen. It’s called “Monte Carlo” because it’s very much like gambling in a casino, except you a4ren’t betting any5thing except your time. You just roll some dice (they might be funny-looking non-cubical polyhedra) or spinning a wheel or throwing darts or spattering paint or vaporized metal… And then you see what happens, and draw conclusions. Today, it’s 4really easy t6o do.
So I decided to see whether, in fact, the number of letters in the teachers’ names had any correlation with their Value Added scores. (I thought it was possible, tho not very likely.) I discovered that Excel found the r-squared constant was about 0.000000. That is zero correlation, my friends. Here is one such scatterplot:
The vertical axis, which goes up the middle, is the number of letter in the teachers’ first name times the number of letters in their last name as listed in the spreadsheet. The horizontal axis, which is at the bottom of the page, is their 2005-2006 value-added score, which can be either negative (theoretically bad) or positive (supposedly good). To me, it sort of looks like bush that hasn’t been pruned in several years – a classic case of no correlation at all.
I asked Excel to draw and calculate the line of best fit. It’s the green, nearly-horizontal line near the center of the graph. Notice the r-squared value: 6E-05, which for all of you innumerates out there, means 0.00006, which is seriously smaller (three orders of magnitude smaller) than 0.05; i.e., one-thousandth as big.
Notice that I’m only using r-squared. Someone objected that i should use just r. If you want, take the square root of all of the correlations I had my computer calculate, and you’ll get r. Compare and contrast.
So, in any case, I definitely did exaggerate.
It all makes sense now.
At first I was a bit surprised that Bill Gates and Michelle Rhee were opposed to publicizing the value-added data from New York City and other cities.
Could they be experiencing twinges of a bad conscience?
That’s not it. Nor do these educational Deformers think that value-added mysticism is nonsense. They think it’s wonderful and that teachers’ ability to retain their jobs and earn bonuses or warnings should largely depend on it.
The problem, for them, is that they don’t want the public to see for themselves that it’s a complete and utter crock. Nor to see the little man behind the curtain.
I present evidence of the fallacy of depending on “value-added” measurements in yet another graph — this time using what NYCPS says is the actual value-added scores of all of the many thousands of elementary school teachers for whom they have such value-added scores in the school years that ended in 2006 and in 2007.
I was afraid that by using the percentile ranks as I did in my previous post, I might have exaggerated or distorted how bad “value added” really was.
No worries, mate – it’s even more embarrassing for the educational deformers this way.
In any introductory statistics course, you learn that a graph like the one below is a textbook case of “no correlation”. I had Excel draw a line of best fit anyway, and calculate an r-squared correlation coefficient. Its value? 0.057 — once again, just about as close to zero correlation as you are ever going to find in the real world.
In plain English, what that means is that there is essentially no such thing as a teacher who is consistently wonderful (or awful) on this extremely complicated measurement scheme. How teacher X does one year in “value-added” in no way allows anybody to predict how teacher X will do the next year. They could do much worse, they could do much better, they could do about the same.
Even I find this to be an amazing revelation. What about you?
And to think that I’m not making any of this up. (unlike Michelle Rhee, who loves to invent statistics and “facts”.)
I neglected to give the links to where you can find the raw data. (Warning: some of these spreadsheets are enormous); Here they are:
Check out the email someone forwarded to me from a California lecturing group (MPSF) that has booked Michelle Rhee. Apparently they are afraid of big, bad teachers coming after her.
Here is the website for the lecture series if you want to make their
dreams nightmares come true. I think it’s important for the public to realize that MR is an utter charlatan, profiting from destroying public education for our neediest citizens.
Anybody in the DC area interested in seeing “The Inconvenient Truth Behind Waiting For Superman”, which points out how clearly flawed the pro-Rhee propaganda film “Waiting for Superman”?
If you are, let me know. I’ve got a copy, and more folks should see it.
As usual, it’s easy to leave comments on this blog, but you have to look hard to find the button for comments: it’s minuscule.
A comment by ‘Lisa’:
Interesting that this was published in Forbes magazine which is a national business magazine. When reading the article you have to remember that when it says “reform” it really means “deform”!
As a result, I added a few alternative wordings here and there. — GFB
by E.D. Kain
Teachers’ unions are often portrayed by their opponents as standing in the way of efforts to reform [deform]our schools. Maybe this isn’t such a bad thing.
In February of 2011, tens of thousands of protesters descended on the capital, Madison, to protest an extreme union-busting bill aimed at reducing the power of that state’s teacher unions. Ultimately those protests failed, but they were a remarkable display of worker solidarity and middle class organizing. Largely, this was because they involved teachers who remain one of the best organized workforces in the country.
More recently, in Tacoma, Washington teachers went on strike in order to secure slightly smaller class sizes and rebuke a pay cut aimed at closing a shortfall in the district budget. The strike ended recently, with both sides getting some of what they wanted.
I’ve wrestled a great deal with the question of organized labor, especially in the realm of public education. There’s a strong contingent on the right and the left that believes that essentially all of the flaws in our public school system stem from a combination of government inefficiency and union recalcitrance. Some people in the reform movement believe that the o nly way to affect reform is to sidestep or abolish teachers unions.
Many left-leaning [left-sounding] school reformers [deformers] believe this largely as a last resort. These are typically liberals who believe in public education but have become frustrated with what they see as union resistance to much-needed reforms [poorly-thought out fads]. There is no doubt that unions oppose many reforms [fads]. The question is, should they? Much of the time, I think they should.
Many right-leaning reformers [deformers], and some liberals as well, are simply in the business of union-busting, seeking to dismantle a powerful political opponent while ushering for-profit schools in through the side door, and handing out lucrative contracts to political allies in the private sector. For a good example of this second camp, go to Scott Walker’s Wisconsin. Or follow Michelle Rhee around the country – to Florida, Nevada, or voicing her support for union-busting in Wisconsin.
Right or left, this is essentially the neoliberal approach to school reform. Technocratic, choice-based, with a troubling dose of private, for-profit groups thrown into to the mix. Mayor Bloomberg’s New York schools are a perfect example of technocratic, anti-democratic leadership at the top, coupled with private contractors, high-stakes testing companies, and union-busting advocacy groups working from the ground up.
The Unreliable Promise of School Reform
Of course, the word “reform” when juxtaposed with “education” is not in and of itself a bad thing. I believe that some ideas in the school reform movement have potential. There is a real chance that experimenting with how schooling is delivered could have some real benefits.
If we can find ways to deliver education to the most underserved communities in this country, that alone would be a huge step forward. And as I’ve mentioned before, I don’t find every argument on the “anti-reform” side all that compelling either. Nobody is on the side of angels all the time.
Then again, a good charter school is more like a band-aid than it is a structural fix. Worse, some charters could serve as Trojan horses, chartered with the best of intentions, but then funded by donors with an anti-union agenda, and paraded in the media as reformer avatars.
A lot of money goes into these schools – quite a lot more than was being funneled from private coffers into the traditional public schools. This says nothing about the people running these schools who are more than likely doing it for the right reasons, but why have these very deep pockets suddenly opened up not to traditional public schools but to the charter school movement? Surely there’s a reason.
Nor do all school choice efforts live up to their promises. Vouchers have been met not only with public disappointment, but with few if a ny real benefits. Most charters haven’t fared much better. And for-profit schools come packaged with all sorts of other troubling implications for the future of our public – or should I say “public” – education system. Do we want to transform our elementary and high schools into little dopplegangers of College of America or the University of Phoenix?
Other ideas are equally repellent, such as high-stakes testing which seeks, in spite of claims to the contrary, to boil down education to learning by rote. When technocrats and businessmen take over the hard work of designing a system of education is it surprising that the result is a complex labyrinth of testing schemes aimed at only those subjects that can be measured, quantified, and pasted in to spreadsheets? Is it any wonder that professionals who care about education might see this as a threat?
Teacher Buy-In is Necessary for Sustainable Reform
If I could wave a magic wand I would graft Finland’s model of public education onto our own. And yes, I realize that even drawing close to Finland’s educational successes will take far more than any magic wand could provide.
One problem we face is that our reform movement has become defined by a very specific, narrow set of ideas: choice and testing and tinkering with teacher compensation and benefits. Very little attention has been paid to curriculum, infrastructure, or equitable school funding.
Most importantly, the very, very hard work of education reform will require teachers if we want it to succeed, whereas the current crop of reformers is intent on bypassing the teachers and especially the teachers’ unions. There are not many advocates for our public education system or for the welfare of children who have the organizational structure and commitment that America’s teachers have.
Parents are the only comparable demographic but they are fragmented and are often only temporary activists. No other group comes close.
Big charities, foundations, and other reform groups can leave the field at any time. They have no vested interest save their own interest, for better or worse.
The National Education Association, or NEA, was founded as a professional association of educators in 1857 and evolved into the largest teachers union in the country. The American Federation of Teachers, or AFT, was founded decades later in 1916 and is affiliated with the AFL-CIO. Both organizations have worked to improve school conditions both for teachers and students for decades. This is important to think about. They are not newcomers to the field. They’ve had their hands deep in the dirt for much of the history of public education.
Yes, we hear stories of unions and union members behaving badly, and these stories will be forever regurgitated to prove that the institutions themselves are to blame. What institution is exempt from this critique? What group of people does not have such members?
A sustained effort to improve and reform our schools will not only require the commitment of the labor force in the here and now, it will require the long-term commitment of teachers both in the field now and those who will come later. Some portions of our education system may need to be reformed, this is very true. Reform should never stop. Progress doesn’t magically happen whilst we twiddle our collective thumbs. But without teacher buy-in no reforms will stick.
Furthermore, without the participation of teachers in the reform process, we risk sacrificing . An even lower-paid teaching force, with fewer benefits and less job security, is what the neoliberal approach to education reform actually offers. It’s not what’s promised: reformers often honestly want to make a grand bargain between job security and teacher pay, offering more of the latter for less of the former. But we know how these things work. Once they take away job security and collective bargaining rights, what’s to stop them from taking away pay, benefits, and everything else?
Democracy in Education
The teachers unions are not always right. No group is. But they represent a democratic approach to our public education system, and if we push them out and usher in an age of for-profit online schools, cheaper labor, and funnel all those saved tax dollars back in the pockets of the wealthiest Americans, we may as well kiss our public schools goodbye.
I support teachers unions not because they are a model of efficiency or because they are always right or because I think there is no need for reform – I believe that the unions can be inefficient, they can be wrongheaded, they can oppose change unnecessarily.
No, I support teachers unions because they are the best chance this country has to improve and strengthen public e ducation for the long haul. No other organization will step in to protect teachers from political blowback and the reform-trend-of-the-moment. The Gates Foundation may have its heart in the right place, but the big foundations can’t protect teachers from slashed budgets or political retribution. Charity-propelled education reform may very well be a sincere effort, but in the process its leaders have offered up a lot of bad choices for teachers. Too often charity reform translates into little more than corporate reform.
Teachers are on the front lines of the fight to keep America’s egalitarian system of public education public. Faux privatization schemes and for-profit schools threaten to undermine the system itself in the name of choice. But what about democracy? What about a system built around the ethic of society rather than that of the individual?
Certainly our public school system could be better. Nobody is suggesting otherwise. Maybe there are radical ideas out there that really would work if given the chance, and I do support experimentation which was the original purpose of charter schools. Let’s keep trying things. Let’s keep experimenting, innovating, and learning from our mistakes. But let’s not do this on the backs of our teaching workforce without bothering to include them in the process.
Teachers are one of the last bastions of workplace democracy left in the country, and once they’re out of the picture anything goes. Including public education.Even if you believe that public education itself is wrong and would prefer a voluntary education system, or a system of teacher/parent cooperatives, or an entirely private or voucher-based system, you still need teachers. In any of these systems teachers should still have the right to form a union, to bargain and negotiate for wages and working conditions, and to find voice and agency in the in the loud cacophony of common cause.