People are Not Cattle!

This apparently did not occur to William Sanders.

He thought that statistical methods that are useful with farm animals could also be used to measure effectiveness of teachers.

I grew up on a farm, and as both a kid and a young man I had considerable experience handling cows, chickens, and sheep. (These are generic critter photos, not the actual animals we had.)

I also taught math and some science to kids like the ones shown below for over 30 years.

guy teaching  deal students

Caring for farm animals and teaching young people are not the same thing.


As the saying goes: “Teaching isn’t rocket science. It’s much harder.”

I am quite sure that with careful measurements of different types of feed, medications, pasturage, and bedding, it is quite possible to figure out which mix of those elements might help or hinder the production of milk and cream from dairy cows. That’s because dairy or meat cattle (or chickens, or sheep, or pigs) are pretty simple creatures: all a farmer wants is for them to produce lots of high-quality milk, meat, wool, or eggs for the least cost to the farmer, and without getting in trouble.

William Sanders was well-known for his statistical work with dairy cows. His step into hubris and nuttiness was to translate this sort of mathematics to little humans. From Wikipedia:

“The model has prompted numerous federal lawsuits charging that the evaluation system, which is now tied to teacher pay and tenure in Tennessee, doesn’t take into account student-level variables such as growing up in poverty. In 2014, the American Statistical Association called its validity into question, and other critics have said TVAAS should not be the sole tool used to judge teachers.”

But there are several problems with this.

  • We  don’t have an easily-defined and nationally-agreed upon goal for education that we can actually measure. If you don’t believe this, try asking a random set of people what they think should be primary the goal of education, and listen to all the different ideas!
  • It’s certainly not just ‘higher test scores’ — the math whizzes who brought us “collateralization of debt-swap obligations in leveraged financings” surely had exceedingly high math test scores, but I submit that their character education (as in, ‘not defrauding the public’) was lacking. In their selfishness and hubris, they have succeeded in nearly bankrupting the world economy while buying themselves multiple mansions and yachts, yet causing misery to billions living in slums around the world and millions here in the US who lost their homes and are now sleeping in their cars.
  • Is our goal also to ‘educate’ our future generations for the lowest cost? Given the prices for the best private schools and private tutors, it is clear that the wealthy believe that THEIR children should be afforded excellent educations that include very small classes, sports, drama, music, free play and exploration, foreign languages, writing, literature, a deep understanding and competency in mathematics & all of the sciences, as well as a solid grounding in the social sciences (including history, civics, and character education). Those parents realize that a good education is expensive, so they ‘throw money at the problem’. Unfortunately, the wealthy don’t want to do the same for the children of the poor.
  • Reducing the goals of education to just a student’s scores on secretive tests in just two subjects, and claiming that it’s possible to tease out the effectiveness of ANY teacher, even those who teach neither English/Language Arts or Math, is madness.
  • Why? Study after study (not by Sanders, of course) has shown that the actual influence of any given teacher on a student is only from 1% of 14% of test scores. By far the greatest influence is from the student’s own family background, not the ability of a single teacher to raise test scores in April. (An effect which I have shown is chimerical — the effect one year is mostly likely completely different the next year!)
  • By comparison, a cow’s life is pretty simple. They eat whatever they are given (be that straw, shredded newspaper, cotton seeds, chicken poop mixed with sawdust, or even the dregs from squeezing out orange juice [no, I’m not making that up.]. Cows also poop, drink, pee, chew their cud, and sometimes they try to bully each other. If it’s a dairy cow, it gets milked twice a day, every day, at set times. If it’s a steer, he/it mostly sits around and eats (and poops and pees) until it’s time to send  them off to the slaughterhouse. That’s pretty much it.
  • Gary Rubinstein and I have dissected the value-added scores for New York City public school teachers that were computed and released by the New York Times. We both found that for any given teacher who taught the same subject matter and grade level in the very same school over the period of the NYT data, there was almost NO CORRELATION between their scores for one year to the next.
  • We also showed that teachers who were given scores in both math and reading (say, elementary teachers), there was almost no correlation between their scores in math and in reading.
  • Furthermore, with teachers who were given scores in a single subject (say, math) but at different grade levels (say, 6th and 7th grade math), you guessed it: extremely low correlation.
  • In other words, it seemed to act like a very, very expensive and complicated random-number generator.
  • People have much, much more complicated inputs, and much more complicated outputs. Someone should have written on William Sanders’ tombstone the phrase “People are not cattle.”

Interesting fact: Jason Kamras was considered to be the architect of Value-Added measurement for teachers in Washington, DC, implemented under the notorious and now-disgraced Michelle Rhee. However, when he left DC to become head of Richmond VA public schools, he did not bring it with him.


What Lessons Has DC Drawn From PISA?

Basically, the lessons drawn by those in charge of education in Washington, DC, is to do exactly the opposite of everything being done by nations with high test scores. Valerie Jablow at EducationDC explains the details.

The Real Lesson of Singapore Math!

By now you’ve probably heard that Singapore and Shanghai are the two places on earth with the smartest kids in the entire world. We can see their PISA scores (go to page 5) are right at the top.

Case closed, right? Whatever they are doing in education, we in the US need to emulate that in order to catch up! Common Core! StudentsFirst! Teach for America! Race to the Top! PARCC! Bust those teacher unions! No more recess! All test prep all the time! Charter Schools! Turn the schools over to the billionaires (Gates, Bloomberg, Koch family, Walton family, and their hirelings and shills)!

But wait a second.

Have you noticed that an ENORMOUS fraction of the low-skilled, low-paid people living in Singapore are temporary foreign workers from various parts of Asia and Africa and are not allowed to bring their kids with them? Those kids are raised back in the workers’ homelands by various relatives, far away, and only get to see their parents at long intervals (somebody has to fly somewhere); back home, jobs are even scarcer and worse-paid, so the parents go elsewhere to try support their families.

Now, everywhere in the world, family income is very, very closely linked to children’s test scores in school. It’s one of the tightest correlations there are in the social sciences, as you can see in the simple scatter-plots I have repeatedly shown in this blog over the past 4 or 5 years. (Try using terms like “poverty” “income” and “scores” together in the search box on this page and be prepared to look through a lot of posts with such graphs, from all over!)

If one-quarter to one-third of the population of a country was legally not permitted to have children in the schools, and it was the low-paying 1/4 to 1/3 of the population, then the scores of the remainder of the kids would, quite naturally, be pretty darned good, since the bottom 1/4 to 1/3 of the distribution just got cut off.

If we systematically excluded the poorest quarter or third of our American student population from taking PISA, we know that our scores would be pretty darned high as well.*

Hmm, maybe the leaning tower of PISA hype is falling.



*Let’s remember that this WAS official policy in many states of the USA up until 1865: a large fraction of the population (guess which one!) was forbidden to send their kids to schools at all and it was explicitly forbidden even to teach them to read privately. When Jim Crow was established from the 1870s to the early 1960s, school facilities for Blacks and Hispanics, BY DESIGN of the racist authorities, so inferior to those for whites that they were a national disgrace. Which is why the calls for going back to the good old days should be so infuriating. There WERE NO GOOD OLD DAYS.

USA Today Shows That Many of the Big Test Score Jumps in DCPS Probably Are Due to Fraud

You MUST read this article in today’s USA Today.

The authors have dug deeply into the suspicious rises in test scores at many DC public schools under Michelle Rhee, and they conclude that a lot of those jumps were almost undoubtedly because somebody changed lots of students’ answers from wrong ones to right ones.

The only question in my mind is who actually did the erasures? Was it the principals themselves, afraid of losing their jobs? Their henchmen assistant principals, or the counsellors, under direct unwritten orders from those principals (and afraid of losing their jobs)? Or was it classroom teachers (also afraid of losing their jobs)?

The authors claim that mathematical probability of so many wrong-to-right erasures happening by chance is about the same as that of winning the Powerball lottery.

The article concentrates most of its focus on Noyes Elementary in Northeast DC, because of the sheer volume of suspicious erasures there. However, DCPS administrators refused to allow the reporters to speak officially to any administrators there, and even refused to allow them to talk to the firm which had been ineffectually investigating prior accusations of fraud.

A number of Noyes parents had for years been protesting to the Noyes administration and to DC School Board officials that their own children’s math and reading scores were suspiciously high. They didn’t feel that their children should be scoring in the “proficient” range if they were struggling with basic reading and writing. They were given the run-around by both the local school administration and by central DCPS administration, and eventually gave up. Some teachers quoted in the article confirmed that for many of their students, the test scores were obviously grossly inflated. The high scores, in any case, didn’t last once the students left the school.

Don’t get the impression that all of the problems happened at Noyes. Many schools had enormous jumps in scores, followed by large cash bonuses for administrators and teachers. Many of those same schools had huge numbers of suspicious erasures.

Also: don’t assume that current or past DCPS administration has any intention of actually digging down into this matter and finding out what really happened. They have been stonewalling this for years.

Meanwhile, at Sharpe Health special education school, and at St. Coletta’s Special Education charter school, well over 90% of the students scored proficient or advanced in both reading and math in the 2010 DC-CAS.

I guess pigs do fly after all.

The Secret to Raising Test Scores? (Part 2)

As I have pointed out, one of the easiest ways to improve test scores in a class or a school is to get rid of the students who generally score low, and to retain the ones who do well. I suspect that may have been the secret behind the fact that Michelle Rhee, after two years of complete and utter failure with her second-grade classes, finally managed to raise the scores to somewhere near the 50th percentile mark.

It appears that the ‘weeding-out’ was pretty drastic.

The cohort that started the first grade at Harlem Park in 1992-1993 had 84 students, probably 3 or 4 distinct classes.

When they arrived in the second grade in 1993-1994 and endured Michelle Rhee’s second failed year of teaching, they still had 83 students – probably 3 or 4 classes again.

But when this cohort arrived in the third grade in 1994-1995, Rhee’s “miracle year”, their numbers dropped by nearly half, to only 44 students. I doubt strongly that so many students dropped dead. I can’t prove it, but I would not be surprised if the school (and Rhee) ‘counseled out’ the ones who were doing poorly, and kept the ones who had high test scores.

You may be wondering if the same thing happened at other Edison schools and at the regular Baltimore public schools that were used for comparison purposes. Look for yourself at the graphs below – and I think you will conclude that while some of the cohorts in some of the schools had small rises in populations, some went down a little bit, and some stayed about the same. NONE OF THEM HAD SUCH HUGE DROPS AS WE SEE IN MICHELLE RHEE’S “WONDER YEAR.”

(By the way, this trick doesn’t always seem to work. The first grade in 1991-2 at Harlem Park had 102 students, and scored at the 63rd percentile in math in the spring. The next year, they were promoted to the second grade, and only had 77 to 84  students, depending on which count you follow, and they had Rhee as one of the second grade teachers. The average percentile rank of this cohort fell from the previous (respectable) 63rd percentile to an abysmal 18th percentile. As someone might say, echoing Michelle Rhee’s recent interview with Harry Jaffe,

these kids were getting screwed because people wanted to blame their low achievement levels on the single-parent households and on the poverty in the community. In that two-year period, none of those things changed. Their parents didn’t change. What changed? What we were doing with them in school.”

Well, if she wants to take credit for a 20-percentile rise in scores from 1994 to 1995, then she needs to take the blame for a 45-percentile-point DROP from 1992 to 1993, even with the weeding out.)

As has been the case for my past several data-driven posts, all of these tables and graphs were made by me from the tables in the appendices to the UMBC Evaluation of the Tesseract Program in Baltimore City, written by Lois C. Williams and Lawrence E. Leak in 1995. You can find it on the web at this URL.

%d bloggers like this: