An Evaluation of Billionaire Gates

Anthony Cody has done a masterful evaluation of Billionaire Gates and the lousy results he’s been getting as he throws his unaccountable billions around in American education. I hope he won’t mind me re-publishing it in its entirety.

Accountability for Mr. Gates: The Billionaire Philanthropist Evaluation

By Anthony Cody on April 5, 2013 12:24 AM

Bill Gates, who is more responsible than anyone for the absurd evaluations by which teachers are now being held accountable, had the gall to write this week in a tone of exasperation about the results of his own advocacy for these very practices.

Yesterday I asked when Mr. Gates, the great enthusiast for accountability for others, might hold himself accountable for his own handiwork.

As wealth has concentrated in the accounts of individuals such as the Gates, Walton and Broad families, they have used this to wield unprecedented power over the lives of those of us without access to such resources. They pay for research that creates the very “facts” upon which public debate is based. They pay for their own media outlets, and heavily subsidize others. Their money redirects existing grassroots groups, and underwrites new ones. They work with ALEC to write legislation, and funnel money through PACs to buy off politicians to move it forward across the country. They are utterly insulated from any sort of accountability. They do not face voters in any election. Nobody “evaluates” them. They cannot be fired. They may on occasion choose to engage in a dialogue, but they are not obliged to respond to the substance of the criticisms raised. As my question indicated, this accountability they demand from teachers is a street that goes one way only.

But let’s imagine we could turn the tables on Mr. Gates and evaluate his performance as a philanthropist. Might we establish some goals to which we could hold our billionaires accountable? We do not have any measurable indicators such as test scores to use, but since I do not find these to be of great value in any case, I will offer a more qualitative metric, based on my knowledge of the subject’s work. Since he has spoken glowingly of the salutary effect of feedback on teachers, surely he will welcome this feedback, even though it is unsolicited.

In the tradition of the Danielson and Marzano teacher evaluation frameworks, I offer the Cody Billionaire Philanthropist Evaluation Model, as applied to Bill Gates.

Standard 1: Awareness of the Social Conditions Targeted by Philanthropy
RatingBelow Standard
Mr. Gates does not demonstrate an understanding of the social conditions that are the focus of his philanthropy. Actions and statements by him and his representatives indicate ignorance of the pervasive effects of poverty, and the overwhelming research that indicates the need to address these effects directly. Mr. Gates has not attended public schools, nor worked in an educational context, and thus he has no personal expertise. He primarily cites research he has paid for himself, which tends to conform to his views. His representatives claim their Foundation lacks the resources to address poverty, and insists that educators bear the burden for overcoming its effects with minimal support.

Recommendation for Professional Growth:

We recommend Mr. Gates take a year off from his work as a philanthropist, and work as a high school instructor in an urban setting. His students should include English learners, students who are homeless, and those designated as Special Education. He should work alongside a fully credentialed professional educator, who will provide him with feedback, and reflect with him as he gains an understanding of how we create effective learning conditions for students.

Standard 2: Understanding of how Learning is Measured
Rating: Below Standard
Mr. Gates has concluded that measurement is the primary means by which social progress can be made. He has determined that test scores are an adequate means of measuring learning, and promoted a wide variety of ways by which these scores are used to measure learning, and reward teachers and students accordingly. This is based on a fundamental error. In fact, test scores measure only a small part of what we value.

Recommendations for Professional Growth:

Mr. Gates should first read Stephen Jay Gould’s Mismeasure of Man, for an understanding of the history of testing. He should also read Daniel Koretz’ book, Measuring Up, What Educational Testing Really Tells Us.

Mr. Gates should, with the help of an experienced educator, design a series of rich PBL projects that allows each of his students to demonstrate their learning through authentic products in real-world contexts. He should compare the work they are capable of producing to their standardized test scores, and reflect on the things that each mode of measurement captures.

Standard 3: Understanding of How Teaching is Evaluated
Rating: Below Standard
Compounding the fundamental error regarding the measurement of learning described under Standard 2 above, Mr. Gates has promoted the use of teacher evaluations based in significant part on test scores and VAM systems. Research does not support this use of test scores, and raising the stakes on test scores has promoted widespread teaching to the test. Mr. Gates has made statements that indicate he is unaware of effective evaluation practices, such as the Peer Assistance and Review program and others.

Recommendations for Professional Growth:
Mr. Gates should spend a week shadowing PAR consulting teachers as they work with teachers in Toledo, Ohio. He should review the research on forms of effective evaluation practices.

As recommended above, he should serve as a classroom teacher for a full year, and have his performance rated based on VAM scores derived from standardized tests taken by his students. He should reflect with his colleagues on the validity of these ratings. He should also meet with a peer evaluator to set professional goals at the start of the year, and several times during the year meet with this person to reflect. At year’s end he should compare the models of evaluation he experienced, and reflect on which were of greater validity and value.

Standard 4: Understanding of Effective Instruction

Rating: Below Standard
Mr. Gates has repeatedly stated that he believes we ought to stop spending money on keeping class sizes small, and instead should use that money to provide performance bonuses for teachers. He has also indicated that we should “personalize” learning through the use of computers and videos that allow students to work at their own pace. This does not comport with what we know about child development, or the importance of personal relationships with students.

Recommendations for Professional Growth
Mr. Gates should spend a week shadowing children in elite schools such as the one attended by his own children, and study the way personalization is accomplished. He should then spend a week shadowing children at a Detroit school where class sizes have been significantly increased due to budget cuts, and the pressure of high stakes have focused instruction on test preparation.

In the year he teaches, he should be assigned at least one class no larger than 15, and another no smaller than 38, and reflect on the learning conditions in these two environments.

Summary of Evaluation Results and Recommendations: 
Mr. Gates falls below standards in all four of the areas that were observed. His philanthropic activities should be suspended immediately pending his completion of the recommended professional growth activities.

A panel of expert reviewers composed of students, parents and educators from communities that are the targets of his philanthropy should be convened to review his reflections at the end of his year of investigation and reflection. This panel should subsequently review and approve the re-initiation of philanthropic projects following this process.

This is the beginning of what might be a far more complex process of reflection for Mr. Gates. It might be seen as absurd, but my intention is sincere. His thinking is magnified in its effect by the billions he has to spend as he chooses. With such power comes a huge responsibility to learn from one’s mistakes. I do not know how Mr. Gates reflects on the successes and failures of his work – there is no evidence of thoughtful reflection in his public writing.

Fairness demands that accountability cannot be a one way street. If Mr. Gates demands that teachers be held accountable for their work, surely he must accept some accountability for his. What is good for the poor geese ought to be good for the billionaire gander, even if he does lay golden eggs.

What do you think of this feedback? Are there other standards we might use to judge the quality of the work of billionaire philanthropists? Have I been fair with Mr. Gates?

Published in: on April 6, 2013 at 10:52 am  Comments (1)  
Tags: , ,

Problems With The Labor Movement

This column was prompted by an exchange on the ConcernedForDCPS list-serve. I’m going to quote a bit here:

(1) RN wrote: “Things are looking worse for Black male high school dropouts.”

(2) RL responded: “so what do you suggest?”

(3) HT responded, in part, and in his usual nearly-inscrutable prose:  

“RN,  There’s nothing to stop you from posting here, and regularly, on the efforts and increases in efforts to find more effective ones, by the metro-area labor council and of individual unions, all to make a bigger difference. Just as Rhee and Henderson, squat is what I see. (Let’s call it as it is, after Easter.) “

Me again:

To continue the digestive analogy, the only way to produce something worth “squat” is to have a “movement” — and in our case, the “movement” needs to be one in which there are LOTS of people who feel strongly enough about to join and march on the streets and go to meetings and boycott and donate and go on strike. And such a movement also needs a leadership willing to devote essentially all their time to it, to the detriment of the rest of their lives.

(It’s quite different if you ‘devote’ your life to the Dark Side: you can easily make 6 figures for showing up in business wear, attending well-catered meetings in comfort, manipulating suspect “data” on spreadsheets, and writing inane drivel using whatever the latest corporate jargon is required this week.  If you join the Dark Side, the side of the one-hundredth-of -one-percent, the side of the billionaires and the Global Corporate Edu DEformers like Michael Milken, Bill Gates, and Rupert Murdoch,  you can look forward to 7-, 8-, or 9-figure salaries even if you aren’t the most TV-pleasing face out there. But you will be rich, as Michelle Rhee once told me and my fellow-teachers from my school who came to see her to complain about our local situation, “Rich Beyond Your Wildest Dreams”. And she said it like it was the last line in a fairy-tale come true… )

And it was the last line of the meeting before we teachers were dismissed (to a towed car, in one case).

I just wish I had done enough research on her shady history in Baltimore at the time to call her on it at the meeting.

One of the big problems is that we keep getting sold out by our own union leadership, over and over. We here in DC’s WTU local have had a horrible experience for decades now: one disastrous leader, then another, then a thieving crook, then a complete and utter traitor, followed by a nice enough guy who doesn’t seem to quite have enough fire under him.

I hope he’s not a disaster, a crook, or a traitor, but when does the public ever find out about that stuff? When it’s too late and the wrong type of smelly ‘movement’ hits the fan! Obviously I’m not a very good judge of character, because I never thought that any of those @#$%^’s would go that far, and I feel like I’m utterly inept as a leader of anything larger than a classroom of math students (and not always even then).

We also have a national WTU leader who has never, ever taught in a classroom.  Despite her occasional militant rhetoric, she trained as a lawyer. She was instrumental in shepherding in the “cage-busting” contract between Chancellor Michelle Rhee and Local 6 of the WTU, which directly led to the current whirlwind of labor givebacks and union-busting all over the country, along with condemning millions of inner-city kids to a stupid year-long regime of test prep for stupid tests that don’t help anybody except the corporations who print and score them and make tons of cash. Weingarten also has co-sponsored a number of things with, and sits on boards with, people like Rhee and Gates…

This sort of sell-out, criminal leadership that doesn’t share the interests of the rank-and-file, or of the public, is not new in the labor movement. Look it up in any history thereof. But there have been others who were great and who never sold out and achieved great success. They and the millions who marched with them directly or indirectly bequeathed us the Weekend and the 40-hour week; Medicare, Social Security, integrated work places, unions, pensions, integrated schools, Worker’s Comp,  Unemployment Benefits and OSHA… and an end to Jim Crow, and eliminated child labor… mostly. All of which are under attack today.

Guy Brandenburg, Washington, DC

Published in: on April 1, 2013 at 4:21 pm  Comments (1)  
Tags: , , , , ,

What Will the Frontline Report on Michelle Rhee Be Like?

From: Marilyn Williams:
   In case you are interested, Frontline, Michelle Rhee’s Legacy will air on Tuesday, 1.08.2013 at 10pm PBS.
Spread the word.

GFB here:
This bears watching. My TV is set to record the parts I will miss. Thank you, Marilyn, for bringing this to attention or reminding us.
Sounded to me when I talked to John Merrow a couple of times, some months ago, that this version of Frontline won’t be kind and fawning to Michelle Rhee and the entire corporate educational DEform movement* as many  as many thought the original was.
Maybe the tide will turn against this nonsense sooner than I expected.
There have been many, many ridiculous “reforms” that have been foisted on public education since, say, the 1800s, and most of them have been pretty stupid, though well-intentioned. The current Corporate Educational Movement, with Michelle Rhee as its ‘poster girl’, looks like one of the most stupid *** fad or movement ever foisted on public school students and their teachers. In my opinion, the current fad is having the worst and most widespread pernicious effects of any that I can recall either from living and working through them, or from reading and hearing about them from my elders. It is actually having tremendous success in dismantling public education, especially since the a state Supreme Court just ruled that charter schools maybe are   or, according  to the NLRB,  are not in the public sector at all.
I don’t remember the original series well enough to recall exactly what I thought when I saw them, but I do remember the part where Rhee said something like this (as I recall it — someone else can look up the exact words and correct me where my memory twisted things – as does the memory of every other human being on earth):
Interviewer: Ms. Rhee, have you done anything you later on regretted doing?
 {with the implication that this was a softball, open-ended question that she could interpret any way she wanted and, say, described a case where she had made a mistake, and then follow up by explaining how she was able to fix it by working harder; obviously one area where there had been a lot of bitterly-opposed actions by her might be fair game, right? So she might decide to concede one error to show she’s human? Not Michelle Rhee.}
Rhee: [Serious, not joking at all.} You know, unlike anybody else I know, in my entire life I have never done a single thing that I regretted. Ever.
I don’t think she was joking.
If I am correct, and Rhee was dead serious, then what kind of crazy egomaniac are we dealing with anyway? Why has this crazy person apparently been anointed by the wealthiest people in the country to be in charge of determining the route that education in this country**? Why isn’t she a candidate for mental health treatment instead?
 Will this version of Frontline apologize and excuse and gloss over the complete and utter failures and very profitable frauds of Michelle Rhee and her corporate educational DEform* paymasters**? Or will Merrow point out a lot of those lies, failures, and frauds?
* (Also called GERM: Global Educational Reform Movement), by Pasi Sahlberg and others.
** Joel Klein, Michael Bloomberg, Bill Gates, the Koch Brothers, the people behind ALEC etc etc etc.
*** stupid in the sense that every single one of the centrally-written tests that this entire movement is based on, are, risibly and obviously, stupid. Yeah, that’s right. They are stupid tests written by overworked, underpaid, temporary workers while the company rakes in billions in state, local, and federal payments and fees. These tests, which bear almost no connection to concepts that are worth learning, are the ones that my colleagues remaining in the classroom are legally required to administer, and who are judged on some utterly arcane statistical formula that has never been explained to the public or even to any individual teacher who has questioned his or her own rating: a VAM of unbelievable and incomprehensible complexity.


Your thoughts? (You have to click on the tiny “comment” button below – it’s unfortunately very hard to see.)

If the tests by which all of education is to be measured by are garbage, then so are the results

On this blog I have reprinted examples of what I see are crappy test items and dissected them, hoping to show readers that those items neither made sense nor measured what they are purported to measure.

However, I never worked inside the testing industry itself, so I don’t have direct experience of making up BS test items on an industrial scale.* My own experience, however, is that EVERY test — no matter how good — has validity and reliability problems. This passage shows that the tests on which all US educational decisions are supposed to be based are, in fact, ridiculously badly made from the beginning, and cannot possibly measure what they pretend to measure, are unreliable, and thus utterly invalid.  (Plus the tests are snatching at least potentially valuable class time away from our students, while enabling a handful of big corporations like Pearson (more on which below) are raking in huge dividends because they control almost the entire education market.)


This comes from an interview published by Diane Ravitch ( )

Rebecca Rubenstein: Since your book was published in 2009, has the “standardized” testing industry improved?

Todd Farley: Not the slightest bit. There was a story in The New York Times in 2001 about how test-scoring was a wildly out-of-control industry, which quotes various employees—not me!—as saying that they faced “too little time, too much to do, not enough people.” It implies the industry was doing a terribly suspect job. Since then, the industry is about a hundred times bigger, but those problems mentioned in the Times article or in my book have never been addressed. The industry has simply grown exponentially, and there are hundreds of millions of dollars to be earned by companies that are completely unregulated—to repeat, completely unregulated, so whatever Pearson et. al. tell us, we’re supposed to say “thank you very much” and just write them a staggeringly large check—but of course things haven’t gotten any better.

In my time in test-scoring, we never had enough temporary employees to do the work; we always had too much to do and too little time to do it; and there were always financial punishments looming over our heads if we didn’t get things done. We cut whatever corners we could to get it done (I’m sorry to say). Today the work load is a hundred times bigger and the money to be made is a hundred times bigger, but the system didn’t work to begin with and of course it doesn’t work now.

The same is true in the test development business. When I worked for one publisher as a test developer, it was always a madcap race to get tests written on time, and we faced absurd deadlines and pressure to do so. The reality is that quality was always secondary to the bottom line when developing tests, and then when the Common Core standards were introduced, and tests and products needed to be written for them, our deadlines became laughably absurd; I was once involved in the development of 200 tests in two months, which I think is literally more tests than ETS has produced in its entire existence. With the Common Core standards released, all the companies knew all the other companies were racing to finish their tests and products first, so quality became even worse than secondary. It became tertiary, or “fourthiary,” or whatever. Subcontractors who had been fired for poor work were rehired; item writers were hired off Craigslist; test developers with neither teaching experience nor test development experience were given full-time jobs. It’s important to remember that at the end of the day, companies like Pearson are for-profit enterprises. They want to make money. They want to make money, so of course they do a crappy job, because the quality of the work is never anywhere near as important as their desire to make a profit, and there’s always too much work and too little time to do it.

continue reading …

A comment: I was at first skeptical of the “200 tests” mentioned being more that the ETS has created in its entire existence. But I think he may be right: The SAT is essentially one, or two, or three tests, depending on how you look at it; it just gets revised a little bit each year. Reading, Math, and Writing. Plus, there perhaps a couple of score different Advanced Placement (AP) tests and Achievement tests in different subjects; they get revised every year, at least they do in the field of math (which I follow, of course) and others.

But what Pearson is doing now is essentially trying to replace the teacher in every single grade level, for every single course, by making the entire curriculum driven by the tests and pre-tests and practice tests and test prep material provided by them.  Yes, I do mean all of third grade. Yes, I do mean 6th grade science, music appreciation, and geography and PE. Every class. And if you count every single course or subject area that a student might be measured by from Pre-K-3 all the way up to graduating from high school, that might in fact be roughly 200 brand-new test series! Not just end-of-course tests, by no means. A different corporate multiple-choice test every month or two!

All this corporate educa-crap is just that: crap forced down the throat of public school kids and ONLY kids in public schools.

And it won’t improve a damned thing. Except for corporate bottom lines.

Of course the children or grandchildren of Michelle Rhee, Michael Bloomberg, Arne Duncan, Eli Broad, Bill Gates, the Koch brothers, and Barack Obama will never, ever be subjected to such a poor excuse for an education.

Oh, no.

That’s just for the poor black and latino and white kids who are in high-poverty regions; the only way they can opt out is to go to a charter school which might be doing any damned thing and is almost sure to be even more segregated than the nearest public school, if that’s even possible.

This is progress?


* My students and I often found mistakes on tests and quizzes and assignments I made up. I used to congratulate the student and give him/her/them a point when they pointed out an error. ETS and Pearson’s responses have been rather different. Remember the famous talking pineapple question? And do you recall that essentially no-one has ever been able to explain, line by line, number by number, exactly how ANY single teacher’s VAM numbers were calculated? Has any school district ever released data showing how well VAM and supposedly ‘scientific’ classroom observation data correlate with each other? (Hint: they don’t!!)

Once again, let me urge the leadership of the Washington Teachers’ Union, and teacher unions elsewhere, to enlist a good statistician with his/her feet on the ground, and poke holes in VAM. It’s all a tissue of fabrications.

Now I Understand Why Bill Gates Didn’t Want The Value-Added Data Made Public

It all makes sense now.

At first I was a bit surprised that Bill Gates and Michelle Rhee were opposed to publicizing the value-added data from New York City and other cities.

Could they be experiencing twinges of a bad conscience?

No way.

That’s not it. Nor do these educational Deformers think that value-added mysticism is nonsense. They think it’s wonderful and that teachers’ ability to retain their jobs and earn bonuses or warnings should largely depend on it.

The problem, for them, is that they don’t want the public to see for themselves that it’s a complete and utter crock. Nor to see the little man behind the curtain.

I present evidence of the fallacy of depending on “value-added” measurements in yet another graph — this time using what NYCPS says is the actual value-added scores of all of the many thousands of elementary school teachers for whom they have such value-added scores in the school years that ended in 2006 and in 2007.

I was afraid that by using the percentile ranks as I did in my previous post, I might have exaggerated or distorted how bad “value added” really was.

No worries, mate – it’s even more embarrassing for the educational deformers this way.

In any introductory statistics course, you learn that a graph like the one below is a textbook case of “no correlation”. I had Excel draw a line of best fit anyway, and calculate an r-squared correlation coefficient. Its value? 0.057 — once again, just about as close to zero correlation as you are ever going to find in the real world.

In plain English, what that means is that there is essentially no such thing as a teacher who is consistently wonderful (or awful) on this extremely complicated measurement scheme. How teacher X does one year in “value-added” in no way allows anybody to predict how teacher X will do the next year. They could do much worse, they could do much better, they could do about the same.

Even I find this to be an amazing revelation. What about you?

And to think that I’m not making any of this up. (unlike Michelle Rhee, who loves to invent statistics and “facts”.)


I neglected to give the links to where you can find the raw data. (Warning: some of these spreadsheets are enormous); Here they are:–2007-2010-nyc-teacher-performance-data#doereports
or, if you prefer a shorter URL, try this one:

Union Membership and Test Scores

Bill Gates, like many other educational Deformers, makes the assumption that having a strong teacher union is a Bad Thing for students, because the union supposedly runs the evil bureaucracy that does its level best to avoid teaching anything to students.

From my viewpoint, having taught for over 30 years in Washington DC, that assumption is so wrong it’s laughable. At best, the union can negotiate pay scales and can represent teachers (good or bad) so that they get some sort of due process when the school administration goes after them. However, much of the time, teachers are too frightened even to speak up, and even with due process, many teachers do get canned or forced out quietly.

In any case, I wanted to look at a particular paragraph where the supposedly highly intelligent Gates compares Florida and Texas, on one hand, with Massachusetts and New York on the other.

Gates makes the assumption that Florida and Texas, which are “right-to-work” states and whose politicians have famously taken a leadership role in the current educational Deform movement, should have students who do better on various standardized tests.

Here’s the quote:

“Asked if the National Education Association and the American Federation of Teachers have any incentive to back school reforms that help kids but also diminish union power, Mr. Gates responds by questioning the scope of that power. “We have heavy union states and heavy right-to-work states, and the educational achievement of K-12 students is not at all predicted by how strong the union rules are,” he says. “If I saw that [right-to-work states like] Texas and Florida were running a great K-12 system, but [heavy union states like] New York and Massachusetts have really messed this up, then I could draw a correlation and say it’s either got to be the union—or the weather.””

Well, in fact, in general, Texas and Florida generally score LOWER on the NAEP, the only national test we have outside of the SAT and ACT, than do New York and Massachusetts. In fact, students in Massachusetts consistently scores well above all other states in all of the reading and math NAEP tests given in 2009 at the 4th and 8th grade level.

I prepared some graphs using publicly available data comparing 2009 math and reading NAEP scores and percentage of union membership among all employees in each of the 50 states plus DC. I have picked out some of the very highest-scoring states and some of the very lowest-scoring states. (No surprise, when you compare the city of DC versus the states, DC is last.)

Here is a graph showing the 4th grade reading NAEP scores in the y-axis versus the average total percentage of union members, in each state.

Dots that are close to the right-hand edge of the graph represent states with relatively high percentages of workers that belong to labor unions. In each of these graphs, New York has the very highest percentage of union members of all of the states. The state with the very lowest percentage of union members is North Carolina, at about 3.1%, but I didn’t mark it. Can you figure out which one dot represents NC? It’s the dot a bit to the left of the one for Texas.

Dots that are towards the top edge of the graph are the ones with the highest average state-wide 4th grade reading NAEP scale scores, and the ones towards the bottom are the ones with the lowest scores. I labeled the dots for Massachusetts and New Jersey, which are two of the highest-scoring states; they also have pretty high rates of union membership.

The two lowest dots are for Louisiana and the District of Columbia, which also have relatively low percentages of union membership as well.

I drew two red lines. The vertical one represents the median of the percentages of union members among all employees in each state: about 10.8%. So any dot to the right of that line has a higher-than-median union membership level, and conversely for those to the left of that line. The horizontal line represents the median 4th grade scale score for all of the states and DC: 221. Any state scoring above that line has higher-than-median NAEP scale scores for 2009 in reading for the 4th grade, and vice-versa.

When I look at this graph, it almost looks as if higher percentages of union members in a given state seems to be correlated with somewhat higher grades on the NAEP. But the correlation is not very strong.

Here is another graph, this one for 2009 fourth-grade math NAEP scale scores versus union membership in 2009.

Once again, the two lowest-scoring states are ones with low-to-middling percentages of union members: DC and Alabama. The two states with the highest scores are those with high-to-middling percentages of union members: Massachusetts and New Hampshire. NY, FL and TX have middling scores, but are mostly differentiated by their very different percentages of union members. (Frankly, I am not sure how teachers in rural Texas can pay their rent or mortgages without second jobs…)

Here’s another graph, for 8th grade NAEP reading against percentages of union membership in 2009:

Once again, you see that Bill Gates’s assumption that unions are bad for students doesn’t appear to hold water. Both Florida and Texas, states where very few workers belong to labor unions, scored below the median on the 8th grade reading NAEP. Once again, Massachusetts and New Jersey, with higher-than-median memberships in labor unions, scored at or near the top. New York scored about the same as Florida. The bottom scoring states were DC and Mississippi.

Last graph, for 8th grade 2009 math NAEP scores versus union membership:

Again, if anything, the assumption that he made, that union membership is bad for students, is shattered.

Source for union membership is here.

For the various NAEP scores, you have to go to the main data exploration page for the National Assessment of Educational Progress and look around.

Published in: on July 26, 2011 at 11:42 am  Comments (2)  
Tags: , ,

Bad assumptions by Gates researchers on VAM

Valerie Strauss has another excellent column at The Answer Sheet:

(It was written by Matthew Di Carlo, senior fellow at the non-profit Albert Shanker Institute, located in Washington, D.C. It originally appeared on the institute’s blog.)

Bogus Assessment by Gates of “Value Added Measures”

Gates Report Touting “Value-Added” Reached Wrong Conclusion

(provided by Robert Bligh; written by Jesse Rothstein)

(see the parts that I highlighted)

Re-examination of results finds that the data undermine calls
for the use of value-added models for teacher evaluations

BOULDER, CO (January 13, 2011) – A study released last month by the Gates Foundation has been touted as “some of the strongest evidence to date of the validity of ‘value-added’ analysis,” showing that “[t]eachers’ effectiveness can be reliably estimated by gauging their students’ progress on standardized tests” ( . However, according to professor Jesse Rothstein, an economist at the University of California at Berkeley, the analyses in the report do not support its conclusions. “Interpreted correctly,” he explains, they actually “undermine rather than validate value-added-based approaches to teacher evaluation.”

Rothstein reviewed Learning About Teaching, produced as part of the Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project, for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado at Boulder School of Education.

Rothstein, who in 2009-10 served as Senior Economist for the Council of Economic Advisers and as Chief Economist at the U.S. Department of Labor, has conducted research on the appropriate uses of student test score data, including the use of student achievement records to assess teacher quality.

The MET report uses data from six major urban school districts to, among other things, compare two different value-added scores for teachers: one computed from official state tests, and another from a test designed to measure higher-order, conceptual understanding. Because neither test maps perfectly to the curriculum, substantially divergent results from the two would suggest that neither is likely capturing a teacher’s true effectiveness across the whole intended curriculum. By contrast, if value-added scores from the two tests line up closely with each other, that would increase our confidence that a third test, aligned with the full curriculum teachers are meant to cover, would also yield similar results.

The MET report considered this exact issue and concluded that “Teachers with high value-added on state tests tend to promote deeper conceptual understanding as well.” But what does “tend to” really mean?

Professor Rothstein’s reanalysis of the MET report’s results found that over forty percent of those whose state exam scores place them in the bottom quarter of effectiveness are in the top half on the alternative assessment. “In other words,” he explains, “teacher evaluations based on observed state test outcomes are only slightly better than coin tosses at identifying teachers whose students perform unusually well or badly on assessments of conceptual understanding. This result, underplayed in the MET report, reinforces a number of serious concerns that have been raised about the use of VAMs for teacher evaluations.”

Put another way, “many teachers whose value-added for one test is low are in fact quite effective when judged by the other,” indicating “that a teacher’s value-added for state tests does a poor job of identifying teachers who are effective in a broader sense,” Rothstein writes. “A teacher who focuses on important, demanding skills and knowledge that are not tested may be misidentified as ineffective, while a fairly weak teacher who narrows her focus to the state test may be erroneously praised as effective.” If those value-added results were to be used for teacher retention decisions, students will be deprived of some of their most effective teachers.

The report’s misinterpretation of the study’s data is unfortunate. As Rothstein notes, the MET project is “assembling an unprecedented database of teacher practice measures that promises to greatly improve our understanding of teacher performance,” and which may yet offer valuable information on teacher evaluation. However, the new report’s “analyses do not support the report’s conclusions,” he concludes. The true guidance the study provides, in fact, “points in the opposite direction from that indicated by its poorly-supported conclusions” and indicates that value-added scores are unlikely to be useful measures of teacher effectiveness.

Find Jesse Rothstein’s review on the NEPC website at:

Find Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project, by Thomas J. Kane and Steven Cantrell, on the web at:

The Think Twice think tank review project (, a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected think tank publications. The project is made possible in part by the generous support of the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on NEPC, please visit .

This review is also found on the GLC website at

Jesse Rothstein
(510) 643-8561

William Mathis, NEPC
(802) 383-0058

US students are actually doing pretty well, compared to world!

We’ve all heard the continuing refrain from various politicians and “experts” that US schools are failing, that American students aren’t learning anything, that our schools are terrible compared to the rest of the world, and therefore we need to dismantle our public schools by one method or another.


Here is some evidence from the latest international TIMMS scores in the 4th and 8th grades. The data follows.

The red scores are from nations whose students do significantly better than American students.

The part in white shows nations whose students do about the same as American students.

The part in pink shows nations whose students score significantly worse than American students.

2007 timms us comparisonNotice that American students IMPROVED from 4th grade to the 8th grade, in comparison to the other countries – exactly the opposite of conventional wisdom.

And think about whether you would want your child, or any child you know, to have to undergo the pressure-cooker, totally conformist, force-fed type of education that they have in Taiwan, Hong Kong, Japan, South Korea, or Singapore. At least in America, students learn how to question authority and think for themselves. There is precious little of that in other school systems, based on my own personal experience.

Hmm – maybe that’s why we are being bombarded with the idea that Japanese, Chinese, and Korean schools are so wonderful: students there are essentially NEVER expected to question anything, and are trained to exhibit no independent thinking at all. Maybe that’s why powerful corporations and right-wing forces here in the US want to re-make US schools, even though they are, overall, doing a pretty good job?


Get every new post delivered to your Inbox.

Join 420 other followers