Texas Decision Slams Value Added Measurements

And it does so for many of the reasons that I have been advocating. I am going to quote the entirety of Diane Ravitch’s column on this:

Audrey Amrein-Beardsley of Arizona State University is one of the nation’s most prominent scholars of teacher evaluation. She is especially critical of VAM (value-added measurement); she has studied TVAAS, EVAAS, and other similar metrics and found them deeply flawed. She has testified frequently in court cases as an expert witness.

In this post, she analyzes the court decision that blocks the use of VAM to evaluate teachers in Houston. The misuse of VAM was especially egregious in Houston, which terminated 221 teachers in one year, based on their VAM scores.

This is a very important article. Amrein-Beardsley and Jesse Rothstein of the University of California testified on behalf of the teachers; Tom Kane (who led the Gates’ Measures of Effective Teaching (MET) Study) and John Friedman (of the notorious Chetty-Friedman-Rockoff study) testified on behalf of the district.

Amrein-Beardsley writes:

Of primary issue will be the following (as taken from Judge Smith’s Summary Judgment released yesterday): “Plaintiffs [will continue to] challenge the use of EVAAS under various aspects of the Fourteenth Amendment, including: (1) procedural due process, due to lack of sufficient information to meaningfully challenge terminations based on low EVAAS scores,” and given “due process is designed to foster government decision-making that is both fair and accurate.”

Related, and of most importance, as also taken directly from Judge Smith’s Summary, he wrote:

HISD’s value-added appraisal system poses a realistic threat to deprive plaintiffs of constitutionally protected property interests in employment.

HISD does not itself calculate the EVAAS score for any of its teachers. Instead, that task is delegated to its third party vendor, SAS. The scores are generated by complex algorithms, employing “sophisticated software and many layers of calculations.” SAS treats these algorithms and software as trade secrets, refusing to divulge them to either HISD or the teachers themselves. HISD has admitted that it does not itself verify or audit the EVAAS scores received from SAS, nor does it engage any contractor to do so. HISD further concedes that any effort by teachers to replicate their own scores, with the limited information available to them, will necessarily fail. This has been confirmed by plaintiffs’ expert, who was unable to replicate the scores despite being given far greater access to the underlying computer codes than is available to an individual teacher [emphasis added, as also related to a prior post about how SAS claimed that plaintiffs violated SAS’s protective order (protecting its trade secrets), that the court overruled, see here].

The EVAAS score might be erroneously calculated for any number of reasons, ranging from data-entry mistakes to glitches in the computer code itself. Algorithms are human creations, and subject to error like any other human endeavor. HISD has acknowledged that mistakes can occur in calculating a teacher’s EVAAS score; moreover, even when a mistake is found in a particular teacher’s score, it will not be promptly corrected. As HISD candidly explained in response to a frequently asked question, “Why can’t my value-added analysis be recalculated?”:

Once completed, any re-analysis can only occur at the system level. What this means is that if we change information for one teacher, we would have to re- run the analysis for the entire district, which has two effects: one, this would be very costly for the district, as the analysis itself would have to be paid for again; and two, this re-analysis has the potential to change all other teachers’ reports.

The remarkable thing about this passage is not simply that cost considerations trump accuracy in teacher evaluations, troubling as that might be. Of greater concern is the house-of-cards fragility of the EVAAS system, where the wrong score of a single teacher could alter the scores of every other teacher in the district. This interconnectivity means that the accuracy of one score hinges upon the accuracy of all. Thus, without access to data supporting all teacher scores, any teacher facing discharge for a low value-added score will necessarily be unable to verify that her own score is error-free.

HISD’s own discovery responses and witnesses concede that an HISD teacher is unable to verify or replicate his EVAAS score based on the limited information provided by HISD.

According to the unrebutted testimony of plaintiffs’ expert, without access to SAS’s proprietary information – the value-added equations, computer source codes, decision rules, and assumptions – EVAAS scores will remain a mysterious “black box,” impervious to challenge.

While conceding that a teacher’s EVAAS score cannot be independently verified, HISD argues that the Constitution does not require the ability to replicate EVAAS scores “down to the last decimal point.” But EVAAS scores are calculated to the second decimal place, so an error as small as one hundredth of a point could spell the difference between a positive or negative EVAAS effectiveness rating, with serious consequences for the affected teacher.

Hence, “When a public agency adopts a policy of making high stakes employment decisions based on secret algorithms incompatible with minimum due process, the proper remedy is to overturn the policy.”

Comparing Texas Charter and Public Schools

I am copying the entirety of this article. No comments needed from me. How about you? — GFB


Game, Set, and Match—Texas SBOE Member Looks at the Numbers Comparing Charter and Traditional Schools

State Board of Education member Thomas Ratliff, a Republican from Mount Pleasant, has taken a look at the performance data of Texas charter schools and traditional public schools operated by independent school districts, and his findings give cold comfort to charter proponents. Here’s Ratliff’s report on those findings and his conclusions published July 13:

Every year the Texas Education Agency releases the “snapshot” of the prior school year’s academic and financial performance for ISD’s and charter schools. These are the facts from the 2012-13 school year (the most recently released report – released last week). Check them for yourself here: http://ritter.tea.state.tx.us/perfreport/snapshot.

Thomas Ratliff

I offer the following key comparisons between ISDs and charter schools:

Dropout and Graduation Rates:

  • ISDs had a dropout rate of 1.5%, charters had a 5.5% dropout rate
  • ISDs had a 4-year graduation rate of 91%, charters had a 60.6% rate
  • ISDs had a 5-year graduation rate of 92.9%, charters had a 70% rate

Academic Performance:

  • ISDs outperformed charters on 3 out of 5 STAAR tests (Math, Science, Social Studies)
  • ISDs matched charters on the other 2 out of 5 STAAR tests(Reading and Writing)
  • ISDs tested 64.5% for college admissions, charters tested 44.2%
  • ISDs average SAT score was 1422, charters average was 1412
  • ISDs average ACT score was 20.6, charters average was 19.7

Staff expenditures & allocation:

  • ISDs spent 57.4% on instructional expenses, charters spent 50.9%
  • ISDs spent 6% [on] central administrative expenses, charters spent 13%
  • ISDs had 3.8% of employees in central or campus administrative roles
  • Charters had 7.6% of employees in central or campus administrative roles

Teacher salary/experience/turnover and class size

  • ISDs average teacher salary was $49,917, charters average was $43,669
  • ISDs had 15.3 students per teacher, charters had 16.8
  • ISDs had 32.1% of teachers with less than 5 years experience
  • Charters had 75.2% of teachers with less than 5 years experience
  • 24% of ISD teachers had advanced degrees, charters had 17.4%
  • ISDs had a teacher turnover rate of 15.6%, charters had 36.7%


Keep in mind these are statewide numbers and admittedly, there are good and bad ISDs and there are good and bad charter schools. But, at the end of the day, we are talking about the state of Texas as a whole and over 5 million kids and their families.

Here are the conclusions I reach after studying the data and talking to experts, educators and people in my district and across Texas.

1) For at least the second year in a row, ISDs outperformed charter schools on dropout rates, state tests, graduation rates, and college entrance exams. If charters are supposed to be competing with ISDs, they are getting beaten in straight sets (to use a tennis analogy).

2) Charter schools spend more on central administrative expenses and less in the classroom, which leads to larger classes being taught by less experienced teachers.

3) Charter schools pay their teachers $6,248 less per year than ISDs. Many refer to competition from charter schools as a key factor to improving education. I do not see this “competition” helping teachers as some try to claim. The fact is, charters hire teachers with less experience and education to save money. This results in a high turnover rate. Over a third of teachers at charter schools leave when they get more experience or more education. Many times, they go work for a nearby ISD.

In conclusion, when you hear the unending and unsubstantiated rhetoric about “failing public schools” from those that support vouchers or other “competitive” school models, it is important to have the facts. ISDs aren’t perfect, but they graduate more kids, keep more kids from dropping out and get more kids career and college ready than their politically connected competitors. Any claims to the contrary just simply are not supported by the facts and at the end of the day facts matter because these lives matter.

The Inequitable Texas School Funding System Is Declared Unconstitutional

This has the potential to be big. A District Court judge has ruled that the system that the state of Texas is using to finance its school system is systematically inequitable and unconstitutional.

From what I’ve heard from Texans at the Network for Public Education meeting this past vernal equinox, Texas for years had a system where kids in poor working-class and/or Latino or Black towns or neighborhoods had much less spent on their education, so much that the school facilities and so on were markedly — even shockingly — different from the facilities and so on in more wealthy towns and regions.

The link to the decision is here.

I would assume that the current governor Rick Perry (whose gifts to comedians just keep on coming) and the rest of his administration will appeal to the state supreme court, and if they lose again there, they will probably go all the way to the US supreme court.

And with our current band of judges, who think that corporations are people and should have the right to buy as much ‘speech’ as they want, wherever they want, in all the elections, well, the 1/10 of 1% have apparently got the best solid 5-4 majority that money can buy. They may very well rule that it’s perfectly OK for a ‘public’ school system to spend three times as much on kids who live in wealthy school districts as they do on others, as long as [insert legalistic gobbledygook here].

The decision is in a pdf format that I can’t seem to copy as text, only as images, so here is the main conclusion for you to peruse, as pictures. Enjoy.

texas school financing unconstitutional

Published in: on August 28, 2014 at 8:28 pm  Comments (1)  
Tags: , ,

A Rant Against Spiritual Solutions and For Use of Calculators

This rant was delivered by Bill Maher of “Real Time” a couple of days ago. I thought it was pretty good, so I transcribed it for you to read. Here goes:


Last Week, [Texas] Governor Rick Perry announced that he had rented out a 70,000-seat stadium in Houston for something called “The Response”,  which sounds like a home pregnancy test but which actually is, to quote the governor, a “Christian prayer service to provide spiritual solutions to the many challenges we face as a nation.” Or as stadium employees are calling it, “Batshit Day.”

I guess the idea is to get together in a big group and pray all at once, and that way the signal is stronger and God doesn’t lose you when he’s going through a canyon.

But here on planet Reality, may I point out that there is no such thing as spiritual solutions to national problems. If that’s where we are as a country; if our official government policy is “Yee-Haw! Jesus take the wheel!” then we’re dead already.

On his Jesus-palooza website, Perry writes “There is hope for America. It lies in heaven, and we will find it on our knees.” He also says that some problems are beyond our power to solve.

What? I thought we [Americans] were the can-do people. And if Perry thinks that only God  can solve our problerms, then why is he even in government? Why doesn’t he just stay home and light a bunch of candles lke Cissy Spacek’s mom in [the movie] Carrie?

Here’s an opposing view. Not only are our problems NOT beyond our power to solve, but they are actually fairly easy to solve. You have a giant budget deficit, like Perry has in Texas? Raise taxes! Federal tax [rates] haven’t been this low since the 1950’s. And THAT, plus two wars and a recession, are the reason we have a huge deficit.

It’s not because God is angry about the gay kissing on Glee.

It doesn’t require prayer to solve.

It requires a calculator. [Much applause]

Politicians like to say “We need new ideas.”


“New ideas” is just a secular version of “spiritual solutions”: something that will magically fix everything. What new idea is going to fix our health care crisis? A magic pill that makes obese children crap out gold bricks?

We don’t need new ideas. We need the balls to implement the ideas we already know work:

  • Cut corporate welfare
  • Slash the defense budget
  • Tax the rich
  • Support the strong unions that created a middle class in the first place
  • Build infrastructure, and
  • Take the profit out of health care.

By the way, Rick Perry isn’t just talking when he says spiritual solutions. Back in April, faced with a devastating drought [in Texas], Perry did what any solutions-oriented, 21st-century civil servant would do, and he proclaimed a Day of Prayer for Rain.

Because we’re all Ancient Mayans now.

Here’s a map of what Texas looked like back in April [with about 5% of the state in red, “exceptionally dry”], and here’s the Eden that it is today [2 months later: about 65% of the state in red].

In the words of Sister Mary Ignatius, ‘God answers all your prayers, and sometimes the answer is “No”.’

Published in: on June 19, 2011 at 1:53 pm  Comments (1)  
Tags: , , ,
%d bloggers like this: