See Jersey Jazzman use the Gaussian Distribution to Show that Arne Duncan and Mike Petrilli are full of it

Excellent lesson from Jersey Jazzman showing that the old tests produce pretty much the same distribution of scores as the new tests.

old and new tests

He has superimposed the green scores from 2008 on top of the 2014 scores for New York state in 8th grade reading, and basically they have almost the same distribution. Furthermore, a scatter plot shows nearly the same thing, and that there is a nearly perfect correlation between the old scores and the new scores, by school.

old and new tests again

Read his article, which is clear and concise. I don’t have time to go into this in depth.

A 3 minute news segment on the NY lawsuit against Value-Added Modeling

Even-handed video from Al-Jazeera interviewing some of the people involved in the anti-Value-Added-Model lawsuit against Value-Added Model evalutations of teachers in New York state.

You may have heard of the lawsuit – it was filed by elementary teacher Sherri Lederman and her lawyer, who is also her husband.

Parents, students, and administrators had nothing but glowing praise for teacher Lederman. In fact, Sherri’s principal is quoted as saying,

“any computer program claiming Lederman ‘ineffective’ is fundamentally flawed.”

Lederman herself states,

“The model doesn’t work. It’s misrepresenting an entire profession.”

Statistician Aaron Pallas of Columbia University states,

“In Sherri’s case, she went from a 14 out of 20, which was fully in the effective range, to 1 out of 20 [the very next year], ineffective, and we look at that and say, ‘How can this be? Can a teacher’s performance really have changed that dramatically from one year to the next?’

“And if the numbers are really jumping around like that, can we really trust that they are telling us something important about a teacher’s career?”

Professor Pallas could perhaps have used one of my graphs as a visual aid, to help show just how much those scores do jump around from year to year, as you see here. This one shows how raw value-added scores for teachers in New York City in school year 2005-2006 correlated with those very same teachers, teaching the same grade level students in the very same schools the exact same subjects, one year later. Gary Rubinstein has similar graphs. You can look here, here, here, or here. if you want to see some more from me on this topic.

The plot that follows is a classic case of ‘nearly no correlation whatsoever’ that we now teach to kids in middle school.

In other words, yes, teachers’ scores do, indeed jump around like crazy from year to year. If you were above average on VAM one year – that is, to the right of the Y axis anywhere, it is quite likely that you will end under the X-axis (and hence below average) the next year. Or not.

I am glad somebody is finally taking this to court, because this sort of mathematics of intimidation has got to stop.

nyc raw value added scores sy 0506 versus 0607

From Mark Naison – Parent Strike!

 This is from Mark Naison, an author and professor at Fordham in NYC — gfb
Mark Naison
July 23 at 4:51pm
What Our Children and Grandchildren Deserve: No Compromise With Current Education Policies

Every post I get from around the country suggests that the attack on teachers, students and public education shows no sign of letting up. Students are being tested more than ever; great teachers are being given low ratings and driven out of the profession; and whole cities are being turned into all charter districts without evidence that this will do anything to empower students, while at the same time funneling profits to consulting firms and real estate developers. While the worst of the attacks are hitting high poverty schools, no districts are immune from the scripting, the micromanagement and the obsession with test results. This nightmare is occurring in states with Republican governors and states with Democratic governors.

Anyone who thinks that the new ECAA legislation being passed by Congress is going to bring relief is being extremely naive. Those who think than any Presidential candidate will make things better is living in Never Never Land. The momentum of current policies on the state and local level is powerful because it is driven by Billionaire dollars. The same people who are controlling the political process in DC are driving privatization and profiteering in public education at the local level.

The only way to fight back against this is civil disobedience. Parent strikes, Student strikes. Teachers strikes. Test Refusal. And innovative tactics to bring the pressure on those who would destroy students lives and teachers careers. Disrupt meetings. Picket peoples houses. Make those who would make students and families pay the price pay a price themselves.

This is why I am very excited about the formation of the group ParentStrike. And the refusal of United Opt Out to compromise at all with ANY federal legislation that uses standardized testing as the basis of school evaluation and uses federal funds to punish schools, school districts and entire states on the basis of test scores.

Now is not the time to compromise. We are already losing badly. It is the time to disrupt. To confuse. To undermine, To resist

Our children and grandchildren deserve better. Much better.

With Friends Like These …

With Friends Like These…

(public education doesn’t need enemies!)

An assessment by Ken Derstine of the overhaul of ESEA / NCLB / ECAA act. Here is the link:

Weekly Roundup of Educational Resistance by Bob Schaeffer

{As usual, this list is collected and distributed by Bob Schaeffer, not by me.}

The U.S. Senate has joined the House of Representatives in responding to growing, grassroots pressure by voting to overhaul “No Child Left Behind” (NCLB). The bills passed by both the Senate and House reflect widespread rejection of failed top-down, test-and-punish strategies as well as the “NCLB on steroids” waiver regime dictated by Arne Duncan. While neither version is close to perfect from an assessment reform perspective, each makes significant progress by rolling back federally mandated high-stakes, eliminating requirements to evaluate educators based on student test scores, and recognizing opt-out rights. FairTest and its allies will closely monitor the conference committee working on compromise language to make sure the gains remain in the final bill sent to President Obama — the alternative is to keep the yoke of NCLB-and-waivers in place for at least two more years, if not much longer. Meanwhile, organizers in many states are keeping the spotlight on the problems of test overuse and misuse, modeling better practices and winning additional policy victories.

Remember that back issues of these weekly updates are archived at:

National End High-Stakes Testing to Help Fix Public Education: Key Civil Rights Leader
National U.S. Senate Rejects Proposal to Give Federal Government More Say in Identifying “Failing” Schools
National Both House and Senate NCLB Overhaul Bills Allow for Penalty-Free Test Opt Out
National “Race to the Top:” Lofty Promises and Top-Down Regulation Brought Few Good Changes to America’s Schools

Exit Exam on Way Out

Two Small Districts Set Opt Out Records

Opposition Coalesces Against Smarter Balanced Tests

Governor Vetoes Opt-Out Bill; State PTA Pushed for Override Vote

More than 10,000 Young People Who Did Not Pass Grad. Test Recently Received Diplomas

Hawaii Teachers Fight Evaluations Based on Student Test Scores

Why Common Core Tests Are Harmful to Students

Third-Grade Promotion Test Pushes Reading Down Into Kindergarten

Fight to Make Charter School Disclose What Test It Uses for Kindergarten Entry

Test Cuts Came After Thorough Debate

Exam Scores Don’t Tell Full Story of Teacher Preparedness

Time Allocated to New State Tests Cut in Half

Nevada After Testing System Breakdown, State to Hire New Assessment Vendor

New Hampshire Schools Can Replace Smarter Balanced Tests with ACT or SAT

New Jersey
Be Wary of New State Teacher Ratings

New Mexico
Court Rejects Suit Seeking to Strip Pearson’s Common Core Testing Contract

New York
High School Models Authentic Assessment
New York Opt Out Movement Plans to Ratchet Up Actions Against Standardized Exam Overkill
New York Pending NCLB Overhaul Offers Hope to Reduce State’s Testing Obsession

North Carolina State’s Largest District Cuts Back Local Test Mandates
North Carolina Cautions About Test-Score-Based Teacher Pay

Students Can Meet Graduation Requirement with Work Samples in Their Home Language

Questions Mount About Using Volatile Test Results to Evaluate Teachers and Schools
Pennsylvania Teachers to School Board: Standardized Testing is Harming Students

Rhode Island
What Tests Like PARCC Do Not Measure

Teachers School Governor on Testing and Evaluations
Tennessee Local School Board to Take Up Opt Out Resolution

New Test Leading Fewer to Get GEDs

Washington State Testing Revolt Pushes State Into Uncharted Waters
Washington Over-Testing is a Flawed Strategy

“How Many Tests Can a Child Withstand?” — with apologies to Bob Dylan

The Beatings in Education Will Continue Until Morale Improves

Bob Schaeffer, Public Education Director
FairTest: National Center for Fair & Open Testing
office-   (239) 395-6773   fax-  (239) 395-6779
mobile- (239) 699-0468

Important Article Shows that ‘Value-Added’ Measurements are Neither Valid nor Reliable

As you probably know, a handful of agricultural researchers and economists have come up with extremely complicated “Value-Added” Measurement (VAM) systems that purport to be able to grade teachers’ output exactly.

These economists (Hanushek, Chetty and a few others) claim that their formulas are magically mathematically able to single out the contribution of every single teacher to the future test scores and total lifetime earnings of their students 5 to 50 years into the future. I’m not kidding.

Of course, those same economists claim that the teacher is the single most important variable affecting their student’s school and trajectories – not family background or income, nor peer pressure, nor even whole-school variables. (Many other studies have shown that the effect of any individual teacher, or all teachers, is pretty small – from 1% to 14% of the entire variation, which corresponds to what I found during my 30 years of teaching … ie, not nearly as much of an impact as I would have liked [or feared], one way or another…)

Diane Ravitch has brought to my attention an important study by Stuart Yen at UMinn that (once again) refutes those claims, which are being used right now in state after state and county after county, to randomly fire large numbers of teachers who have tried to devote their lives to helping students.

According to the study, here are a few of the problems with VAM:

1. As I have shown repeatedly using the New York City value-added scores that were printed in the NYTimes and NYPost, teachers’ VAM scores vary tremendously over time. (More on that below; note that if you use VAM scores, 80% of ALL teachers should be fired after their first year of teaching) Plus RAND researchers found much the same thing in North CarolinaAlso see this. And this.

2. Students are not assigned randomly to teachers (I can vouch for that!) or to schools, and there are always a fair number of students for whom no prior or future data is available, because they move to other schools or states, or drop out, or whatever; and those students with missing data are NOT randomly distributed, which pretty makes the whole VAM setup an exercise in futility.

3. The tests themselves often don’t measure what they are purported to measure. (Complaints about the quality of test items are legion…)

Here is an extensive quote from the article. It’s a section that Ravitch didn’t excerpt, so I will, with a few sentences highlighted by me, since it concurs with what I have repeatedly claimed on my blog:

A largely ignored problem is that true teacher performance, contrary to the main assumption underlying current VAM models, varies over time (Goldhaber & Hansen, 2012). These models assume that each teacher exhibits an underlying trend in performance that can be detected given a sufficient amount of data. The question of stability is not a question about whether average teacher performance rises, declines, or remains flat over time.

The issue that concerns critics of VAM is whether individual teacher performance fluctuates over time in a way that invalidates inferences that an individual teacher is “low-” or “high-” performing.

This distinction is crucial because VAM is increasingly being applied such that individual teachers who are identified as low-performing are to be terminated. From the perspective of individual teachers, it is inappropriate and invalid to fire a teacher whose performance is low this year but high the next year, and it is inappropriate to retain a teacher whose performance is high this year but low next year.

Even if average teacher performance remains stable over time, individual teacher performance may fluctuate wildly from year to year.  (my emphasis – gfb)

While previous studies examined the intertemporal stability of value-added teacher rankings over one-year periods and found that reliability is inadequate for high-stakes decisions, researchers tended to assume that this instability was primarily a function of measurement error and sought ways to reduce this error (Aaronson, Barrow, & Sander, 2007; Ballou, 2005; Koedel & Betts, 2007; McCaffrey, Sass, Lockwood, & Mihaly, 2009).

However, this hypothesis was rejected by Goldhaber and Hansen (2012), who investigated the stability of teacher performance in North Carolina using data spanning 10 years and found that much of a teacher’s true performance varies over time due to unobservable factors such as effort, motivation, and class chemistry that are not easily captured through VAM. This invalidates the assumption of stable teacher performance that is embedded in Hanushek’s (2009b) and Gordon et al.’s (2006) VAM-based policy proposals, as well as VAM models specified by McCaffrey et al. (2009) and Staiger and Rockoff (2010) (see Goldhaber & Hansen, 2012, p. 15).

The implication is that standard estimates of impact when using VAM to identify and replace low-performing teachers are significantly inflated (see Goldhaber & Hansen, 2012, p. 31).

As you also probably know, the four main ‘tools’ of the billionaire-led educational DEform movement are:

* firing lots of teachers

* breaking their unions

* closing public schools and turning education over to the private sector

* changing education into tests to prepare for tests that get the kids ready for tests that are preparation for the real tests

They’ve been doing this for almost a decade now under No Child Left Untested and Race to the Trough, and none of these ‘reforms’ have shown to make any actual improvement in the overall education of our youth.

ASD Reformers Claim They Can Achieve The Impossible

Gary Rubinstein has watched a conclave of uber school reformers in places like New Orleans and Tennessee, so that you don’t have to.

He discovers some amazing things about their amazing claims of success:

1. They don’t have any secret recipes (other than firing lots of teachers and turning the schools over to private entities)

2. These claims of success are not actually backed up by any data

3. For the most part, these outlandish promises (like going from the bottom 5% to the top 25% in 5 years) are simply advertisements designed to get money

Here’s the link:

Definitely worth reading.

“Math for America” teachers meet with some members Congress and apparently give them some sound advice

During the First National Math Festival here in DC (which I missed), back in April, some Math for America – DC* teachers I know were invited to speak with some Congressmen and Senators. According to the press release I was recently given, my colleagues appear to have given the elected reps** sound advice that may or may not be heeded.

{** including Nancy Pelosi, Harry Reid, Chuck Shumer, Al Franken, Lamar Alexander, Patty Murray, Steny Hoyer, among others}

I quote from the press release, in green and my own comments in black:

“House and Senate leaders, field experts, and MfA DC teachers spent the first hour and a half engaging in dialogue on how the ESEA reauthorization would affect the classroom. Joe Herbert spoke to the adverse effects standardized tests had had on his school and his classroom. David Tansey, a[n] MfA DC Master Teacher, offered criteria that such tests should meet in order to provide instructional value to the teacher and the student.”

{notice the clear implication, which Tansey has spelled out to me in detail on several occasions, that the standardized tests that he and his school are required to administer many, many times a year are of absolutely no use to teachers in figuring out how to help their students learn more stuff, better.}

“Joe Herbert wrote, ‘I spoke of the harmful effects of standardized testing on K-12 education, and of the complete lack of statistical basis for evaluating teachers based on their students’ test scores.'”

While Max Mikulec, one of the other teachers, was initially somewhat awestruck by listening to amusing anecdotes from Senator Al Franken, he …

“…went on to say, ‘As I reflected on the day, my initial reaction of pride and hope turned into a feeling of skepticism and apprehension. You cannot imagine how great I would feel if the nation spent billions more dollars developing math education and math teachers. However, I do not see this happening in an effective way. There are endless debates over what standards should be taught in our schools and what the kids should be tested on. Amid all of the debates, the ones who are losing here are the nation’s kids. In their most formative years, a time where they struggle to find any consistency in their own lives, they are being let down by an educational system that will change several times before they graduate high school. Ev en though all of these powerful and important people say that they support math education and that [they] see math teaching as a real profession, I will not believe them until something is actually done to show their support.'”

In addition, Joe Herbert wrote me the following:

“Another point I made is just how much money gets wasted on these tests. I don’t remember the exact number now, but I looked up how much is spent annually on testing before I went to the event (I remember the number was in the billions), and I made the point that we could increase spending on education by that much money without raising taxes a penny if we got rid of the annual testing mandate in NCLB.

“I know that many liberal groups have been proponents of annual testing because it sheds light on the achievement gap. I noted that NAEP provides these same types of data, but does so using statistical sampling so that we don’t have to test every kid every year.”


*Note: MfA and MfA-DC are as far from the TFA idea as it is possible to be. Unlike ‘Teach for Awhile”, MFA actually gives its members a FULL YEAR of math-content and math-pedagogy classes and student teaching experience, assigns them a mentor, and in return expects them to stay in the city, teaching, in their field for a full five years, and does not pretend to have a one-size-fits-all “no excuses” magic wand that will miraculously reproduce the irreproducible miracle that Michelle Rhree pretended to achieve at Harlem Park Elementary in Baltimore in the early 1990s, magically moving 90% of her students from below the 13th percentile to being over the 90th percentile. Right now, MfA DC teachers are some of the most senior math teachers anywhere in DC, either in the regular public schools or charter schools.

Temps Who Have Never Taught are Grading Common Core Tests

Makoto Rich of the New York Times, Peter Greene of Curmudgacation, and Diane Ravitch all discuss the way that Pearson is running the scoring of the Common Core tests, employing temporary employees who have never taught, at $12-14 per hour, with of course no benefits.

If you’ve worked in a mass-chain fast-food joint, then you know what Pearson wants: mindless uniformity. Isn’t that what parents and kids really, really want from our public schools?

Here’s the link.

A closer look at charter and regular public school enrollments, percentages of students at risk, and percentages of students ‘proficient’

Here is another look at the brand-new data concerning four variables in the District of Columbia schools, about which I wrote a couple of days ago. The difference here is that the dots representing the schools are more-or=less proportional to the size of the student body.

1. Is this a regular public school, or a charter school (blue or red):

2. What fraction of the kids at that school are officially considered to be At Risk? (That’s the scale along the x-axis at the bottom of the page)

3. What is the average percentage of the kids at that school are ‘proficient’ in reading and math on the DC-CAS? (That’s the scale along the y-axis at the left-hand side of the page)

4. How big is the school? (That’s the size of the dot, more or less; the legend is at the bottom left-hand corner of the graph)

Time spent looking carefully at this graph will be well-spent. If you click on it, it will expand.

It will certainly show that charter schools have not revolutionized education for the better in DC: for both types of schools, there remains a very strong, negative correlation between the percentages of kids At Risk and ‘pass’ rates on the DC-CAS.

Note that most schools have between 200 and 500 students and that most of the ones that are smaller are actually charter schools. As I wrote a couple of days ago, the schools with the largest fraction of At-Risk students (say, over 2/3 of the student body) are almost all regular DC public schools.

On the second graph, which is otherwise identical to the first, I’ve labeled some of the larger schools.

fixed bicolor, size of school and at risk vs average dc cas 2014 proficiency, both regular public and charter, dc

Here is the one with names of some of the larger schools, so you can see how individual schools fall on this graph.

(Sorry, I there was not enough room to label every single one, and my non-existent HTML skills won’t allow me to make it so that any of the dots are clickable. If any of my readers know how to do that and would like to offer to make that happen, then please let me know in the comments.)

again fixed and revised names and bicolor, size of school and at risk vs average dc cas 2014 proficiency, both regular public and charter, dc

And here is the entire data table. So you can see where every single school lies on these three dimensions.

(PS: I added a few more names of schools and corrected four other small errors, two pointed out by an alert reader.. 2/22/2015)


Get every new post delivered to your Inbox.

Join 460 other followers

%d bloggers like this: