Just how flat ARE those 12th grade NAEP scores?

Perhaps you read or heard that the 12th grade NAEP reading and math scores, which just got reported, were “flat“.

Did you wonder what that meant?

The short answer is: those scores have essentially not changed since they began giving the tests! Not for the kids at the top of the testing heap, not for those at the bottom, not for blacks, not for whites, not for hispanics.

No change, nada, zip.

Not even after a full dozen years of Bush’s looney No Child Left Behind Act, nor its twisted Obama-style descendant, Race to the Trough. Top.

I took a look at the official reports and I’ve plotted them here you can see how little effect all those billions spent on testing;  firing veteran teachers; writing and publishing new tests and standards; and opening thousands of charter schools has had.

Here are the tables:

naep 12th grade reading by percentiles over time

This first graph shows that other than a slight widening of the gap between the kids at the top (at the 90th percentile) and those at the bottom (at the 10th percentile) back in the early 1990s, there has been essentially no change in the average scores over the past two full decades.

I think we can assume that the test makers, who are professional psychometricians and not political appointees, tried their very best to make the test of equal difficulty every year. So those flat lines mean that there has been no change, despite all the efforts of the education secretaries of Clinton, Bush 2, and Obama. And despite the wholesale replacement of an enormous fraction of the nation’s teachers, and the handing over of public education resources to charter school operators.

naep 12th grade reading by group over time

 

This next graph shows much the same thing, but the data is broken down into ethnic/racial groups. Again, these lines are about as flat (horizontal) as you will ever see in the social sciences,

However, I think it’s instructive to note that the gap between, say, Hispanic and Black students on the one hand, and White and Asian students on the other, is much smaller than the gap between the 10th and 90th percentiles we saw in the very first graph: about 30 points as opposed to almost 100 points.
naep 12th grade math by percentiles over time

 

The third graph shows the  NAEP math scores for 12th graders since 2005, since that was the first time that the test was given. The psychometricians atNAEP claim there has been a :statistically significant” change since 2005 in some of those scores, but I don’t really see it. Being “statistically significant’ and being REALLY significant are two different things.

*Note: the 12th grade Math NAEP was given for the first time in 2005, unlike the 12th grade reading test.

naep 12th grade math by group over time

 

And here we have the same data broken down by ethnic/racial groups. Since 2009 there has been essentially no change, and there was precious little before that, except for Asian students.

Diane Ravitch correctly dismissed all of this as a sign that everything that Rod Paige, Margaret Spellings and Arne Duncan have done, is a complete and utter failure. Her conclusion, which I agree with, is that NCLB and RTTT need to be thrown out.

 

What I actually had time to say …

Since I had to abbreviate my remarks, here is what I actually said:

I am Guy Brandenburg, retired DCPS mathematics teacher.

To depart from my text, I want to start by proposing a solution: look hard at the collaborative assessment model being used a few miles away in Montgomery County [MD] and follow the advice of Edwards Deming.

Even though I personally retired before [the establishment of the] IMPACT [teacher evaluation system], I want to use statistics and graphs to show that the Value-Added measurements that are used to evaluate teachers are unreliable, invalid, and do not help teachers improve instruction. To the contrary: IVA measurements are driving a number of excellent, veteran teachers to resign or be fired from DCPS to go elsewhere.

Celebrated mathematician John Ewing says that VAM is “mathematical intimidation” and a “modern, mathematical version of the Emperor’s New Clothes.”

I agree.

One of my colleagues was able to pry the value-added formula [used in DC] from [DC data honcho] Jason Kamras after SIX MONTHS of back-and-forth emails. [Here it is:]

value added formula for dcps - in mathtype format

One problem with that formula is that nobody outside a small group of highly-paid consultants has any idea what are the values of any of those variables.

In not a single case has the [DCPS] Office of Data and Accountability sat down with a teacher and explained, in detail, exactly how a teacher’s score is calculated, student by student and class by class.

Nor has that office shared that data with the Washington Teachers’ Union.

I would ask you, Mr. Catania, to ask the Office of Data and Accountability to share with the WTU all IMPACT scores for every single teacher, including all the sub-scores, for every single class a teacher has.

Now let’s look at some statistics.

My first graph is completely random data points that I had Excel make up for me [and plot as x-y pairs].

pic 3 - completely random points

Notice that even though these are completely random, Excel still found a small correlation: r-squared was about 0.08 and r was about 29%.

Now let’s look at a very strong case of negative correlation in the real world: poverty rates and student achievement in Nebraska:

pic  4 - nebraska poverty vs achievement

The next graph is for the same sort of thing in Wisconsin:

pic 5 - wisconsin poverty vs achievement

Again, quite a strong correlation, just as we see here in Washington, DC:

pic 6 - poverty vs proficiency in DC

Now, how about those Value-Added scores? Do they correlate with classroom observations?

Mostly, we don’t know, because the data is kept secret. However, someone leaked to me the IVA and classroom observation scores for [DCPS in] SY 2009-10, and I plotted them [as you can see below].

pic 7 - VAM versus TLF in DC IMPACT 2009-10

I would say this looks pretty much no correlation at all. It certainly gives teachers no assistance on what to improve in order to help their students learn better.

And how stable are Value-Added measurements [in DCPS] over time? Unfortunately, since DCPS keeps all the data hidden, we don’t know how stable these scores are here. However, the New York Times leaked the value-added data for NYC teachers for several years, and we can look at those scores to [find out]. Here is one such graph [showing how the same teachers, in the same schools, scored in 2008-9 versus 2009-10]:

pic 8 - value added for 2 successive years Rubenstein NYC

That is very close to random.

How about teachers who teach the same subject to two different grade levels, say, fourth-grade math and fifth-grade math? Again, random points:

pic 9 - VAM for same subject different grades NYC rubenstein

One last point:

Mayor Gray and chancellors Henderson and Rhee all claim that education in DC only started improving after mayoral control of the schools, starting in 2007. Look for yourself [in the next two graphs].

pic 11 - naep 8th grade math avge scale scores since 1990 many states incl dc

 

pic 12 naep 4th grade reading scale scores since 1993 many states incl dc

Notice that gains began almost 20 years ago, long before mayoral control or chancellors Rhee and Henderson, long before IMPACT.

To repeat, I suggest that we throw out IMPACT and look hard at the ideas of Edwards Deming and the assessment models used in Montgomery County.

What a Delaware Politician Really Meant

A very funny critique of what a Delware Edu-Political boss actually meant:

http://transparentchristina.wordpress.com/2012/11/07/last-week-i-posted-a-letter-sent-by-secretary-murphy-to-all-of-delawares-teachers-today-i-will-parse-it-netde/

Published in: on November 20, 2012 at 1:54 pm  Leave a Comment  
Tags: , , ,

An Article By A NYC Charter School Teacher for “PolicyMic”

I thought this was worth reprinting:

==========================

Hi all,

I submitted this article about my experiences re: standardized testing and test-based evaluations in NYC schools to PolicyMic this afternoon.  I hope that it resonates with many of you!  Please feel free to read, comment, and forward as you wish.

And, in honor of Mr. Wellstone (July 21, 1944 – October 25, 2002), who reminded us that laborers and teachers are one in the same, “Stand up. Keep fighting.”

Thanks!
Allison LaFave

http://www.policymic.com/articles/17490/romney-loves-teachers-what-teacher-evaluations-and-tests-mean-for-american-teachers

Romney Loves Teachers: What Teacher Evaluations and Tests Really Mean for American Teachers

During Monday’s final presidential debate, Bob Schieffer spurred a collective American chuckle when he cut off Romney’s long-winded brown-nosing with the knee-slapper, “I think we all love teachers…”

I’d love to believe Mr. Schieffer, but as someone who hails from a family of public school teachers and spent last year teaching third grade in a New York City charter school, I have to say, “Bob. You’re adorable. But America’s teachers haven’t felt loved in quite some time.”

Last spring, my principal corralled our school’s third grade teaching team around a kidney-bean shaped table and apologetically explained that we needed to sign forms acknowledging the weight of our students’ test scores on our end-of-year evaluations. Ultimately, our students’ math and ELA scores would comprise as much as 40% of our annual rating.

Now, I don’t know a single educator who outright opposes the idea of fair evaluations and/or some level of teacher accountability. But as I sat quietly in that little red plastic chair, a voice in me cried:

“You want to evaluate me? Great. No problem.

“But let’s also evaluate the misaligned (or nonexistent) curriculum I was given to plan for my classes.”

“Let’s evaluate the number of chairs huddled around single desks, because there are more students in the room than there were last year, and the copy machine, the one that never works.

“Let’s evaluate the number of students with IEPs that aren’t being adequately serviced, and the number of English Language Learner students sitting voiceless in the back of the room, because they have yet to be admitted into nonexistent ELL classes.

“Let’s evaluate the employers who are smugly underpaying/underemploying my students’ parents or guardians, forcing them to work multiple jobs, likely without ever securing benefits for themselves or for their families. Or the number of students who have lost parents or loved ones due to gang violence, substance abuse, or the labyrinth that is our failing criminal justice system. Or the number of my students who didn’t eat dinner last night.

“Let’s evaluate how many hours of sleep I got last night, because I was not afforded adequate prep time during my 10 or 11 hour day in the building, or how many times I’ve skipped out on doctor’s appointments and family events to be here for my students.

“And, finally, let’s evaluate my motivations for being here — because it sure as hell isn’t for the money.”

Last week, Deborah Kenny wrote an op-ed piece decrying the heavy influence of test scores on teacher evaluations. Kenny rightfully claimed that the practice “undermines principals and is demeaning to teachers” and leaves little room for innovative teaching and learning. She went on to say that test-based evaluations inhibit the “culture of trust” between principals and teachers and “discourage the smartest, most talented people from entering the profession.”

While I agree that test-based evaluations are inherently flawed (when was the last time our politicians, Democrats or Republicans, truly analyzed aPearson test?), I am baffled by Kenny’s ultimate argument. It seems that Kenny bashes test-based evaluations because … wait for it … they make it harder for her to fire teachers she doesn’t like – specifically a teacher whose students performed “exceptionally well” on the state exam.

Teachers aren’t statistics, but they also aren’t part of some school-wide homecoming court. Administrators shouldn’t cast votes for the teachers they like or dislike. They should work to support all teachers who act in the best interest of students.

Ms. Kenny also takes a not-so-subtle jab at teachers’ unions, attacking evil tenured teachers in America, who are clearly exploiting their glamorous roles as K-12 educators. However, unions don’t grant tenure; PRINCIPALS grant tenure. And, moreover, Ms. Kenny, like nearly all charter school administrators in America, likely prohibits her teachers from joining their local union.

As someone who has worked in a non-union school, I can tell Ms. Kenny what violates trust between teachers and administrators. Knowing that you can be fired for your personality.  Knowing that there is a fresh crop of well-intentioned, starry-eyed Teach for America kids who can take your place in the time it takes to make a phone call. Knowing that you will be scorned for using your allotted sick days and guilted into working through lunch, during prep time, and hours after the final school bell rings.

I encourage our presidential candidates (and all Americans) to listen to the voices of practicing teachers, who are so often talked about and around during national education debates.

Says Kelly G., a third grade teacher in Brooklyn:

“These teacher evaluations are complex. I honestly used to think that a teacher could indeed be evaluated and held accountable using test scores. And then I started teaching at school that didn’t allow me to do the kind of teaching I thought needed to be done in order to develop intelligent children. There’s nothing quite like having your teaching micromanaged and then being told it was your fault the kids didn’t achieve exemplary scores on the state exam.

“My kids are capable of so much already. Come in and look at their writing. Listen to their discussions. Watch them solve math problems. Their tests scores will not reflect their growth from the school year. A one shot assessment does not give a good picture of student achievement. Have you read those exams? Have you been in the room during testing? Test anxiety vomiting is a real thing in the third grade. Too bad they don’t evaluate me on sick child comforting and vomit clean up. I’m sure my scores on those evaluations would be proficient.”

In popular media, teachers are cast as heroes or villains. They are either lazy, money-grubbing, ne’er-do-wells or Jaime Escalante, the “teacher savior” of the acclaimed film Stand and Deliver.

The truth is, as in most professions, the majority of teachers lie somewhere in the middle of this spectrum. Such romanticized notions of teaching make great stories, but that’s just it; they are stories that too often exaggerate and obscure the truth. Jaime Escalante spent years preparing his students for the AP Calculus exam, not a few inspired semesters. Does that mean that he was an inadequate teacher during the years he spent honing his craft and teaching foundational math concepts to his students? How would Escalante have been rated under the New York City evaluation system?

In his research paper entitled “Effects of Inequality and Poverty vs. Teachers and Schooling on America’s Youth,” David C. Berliner (Regents’ Professor Emeritus in The Mary Lou Fulton Teachers College of Arizona State University) finds that “Outside-of-school factors are three times more powerful in affecting student achievement than are the inside-the-school factors.”

Consequently, he concludes, “The best way to improve America’s schools is through jobs that provide families living wages. Other programs…offer some help for students from poor families. But in the end, it is inequality in income and the poverty that accompanies such inequality that matters most for education.”

America’s education system is in crisis; of this, we can be sure. But let’s stop blaming the dentists for their patients’ cavities.

Chicago Teachers Lead The Way

Anybody who is interested in promoting the welfare of the 99% of the population who don’t have offshore bank accounts or manage hedge funds should be ecstatic over the apparent success of the members of the Chicago Teachers Union in beating back most of Rahm Emanuel’s educational DEform policies.

A lot has been written about this strike. I am thrilled that over 90% of the teachers voted in favor of the strike, that they stuck together, and that they “stuck it” to arrogant Rahm.

Let us remember that EVERYWHERE that the billionaires’ educational DEforms have been tried, they have failed utterly. I salute the Chicago rank-and-file who stood up.

A few good recent columns by bettwer writers than myself here,  here and here. Or for the full URL, try these:

http://www.tnr.com/blog/plank/107294/can-the-chicago-teachers%E2%80%99-strike-fix-democratic-education-reform#

and

http://www.aljazeera.com/indepth/opinion/2012/09/201291774248929713.html

and

http://www.washingtonpost.com/opinions/eugene-robinson-standing-up-for-teachers/2012/09/17/ad3ee650-00fd-11e2-b257-e1c2b3548a4a_story.html

Published in: on September 19, 2012 at 11:44 am  Leave a Comment  
Tags: , , ,

Over 90% of Chicago Teachers Vote to Approve a Strike!! Wow!!


Published on Labor Notes (http://labornotes.org)

Standing Up to Corporate School Agenda, Chicago Teachers Greenlight Strike

STEVEN ASHBY
    |  JUNE 12, 2012

After four days of ballot-counting, the Chicago Teachers Union announced yesterday that 90 percent of CTU members voted to authorize their leaders to call a strike if negotiations continue to go badly. Photo: Bartosz Brzezinski.
After four days of ballot-counting, the Chicago Teachers Union announced yesterday that 90 percent of CTU members voted to authorize their leaders to call a strike if negotiations continue to go badly.
An overwhelming 24,262 members, or 92 percent of the membership, voted. In a resounding display of unity, 98 percent of those voting authorized the strike. There was not a majority of CTU members at any of the 615 schools who voted “no.”
“We’ve been pushed, and pushed, and pushed—and finally we get a chance to push back,” said a CTU activist as she counted strike authorization ballots this week.
Teachers are angry over large classes, too few social workers and teachers’ aides, a deadening of a curriculum increasingly tailored to standardized tests, and a quarter of all schools lacking a library. They are angry at an unelected Board of Education demonizing them as adversaries.
They are angry that the board, ignoring the pleas of parents and teachers, has systematically moved to close schools and reopen them as privatized charters with a non-union, at-will workforce and policies that punish children with learning disabilities, English language learners, and those with difficulties at home that impede their test-taking abilities.
 
Teachers are angry at Mayor Rahm Emanuel, a Democrat, for imposing unpaid work on them. He has pushed a seven-hour school day plus 10 extra teaching days with no additional compensation, heaping more on top of their already heavy workload. An April 2012 University of Illinois study found that teachers work an average 58-hour workweek.
They are angry over the board’s proposal of a 2 percent raise over five years and the imposition of merit pay, giving principals the power to reward favorites and punish union activists.
As CTU President Karen Lewis said, “We are tired of being bullied, belittled, and betrayed” by the Board of Education.
The overwhelming vote to authorize a September strike should “put an end to speculation about how educators really feel,” said Lewis, who added that “we listen to our members.”
The vote is an “indictment” of Chicago Public Schools administrators, Lewis said. Educators “who actually work in our schools,” not billionaire-backed anti-union lobbyists, are the ones who can improve schools “in partnership” with CPS, she said.
“We are calling on CPS to negotiate with the union in good faith,” Lewis said.
Thirty CTU leaders, all educators working in the schools, painstakingly counted ballots as 12 clergy organized by the interfaith workers’ rights group Arise Chicago took shifts to monitor the count, in an effort to deter anti-union forces from challenging the integrity of the vote. Anti-union elements attacked the ballot regardless.

Vaulting the High Bar

The teachers union had a high bar to reach. Illinois legislators, pushed by Mayor Emanuel and Democratic Speaker of the House Michael Madigan, passed a law last June that imposed the unprecedented requirement that 75 percent of all CTU members must vote to authorize a strike. The standard rule demands 50 percent plus one of those who vote approve a strike. Under the Illinois law, members who did not vote would count as “no” votes.
 
Leaders of the anti-union, billionaire-funded, and misnamed Stand for Children organization propelled the legislation. They lined up support from powerful politicians and lobbyists and pressured the state’s NEA and AFT teachers unions into going along with it. President Lewis signed on, but quickly said language was added into the bill she never agreed to.
 
Stand for Children and other anti-union forces gloated that there would never again be a teachers’ strike in Chicago as the 75 percent threshold would be impossible for the union to reach.
 
CTU proved them wrong.
The vote demonstrated the reach of the union’s seven-month contract campaign, an achievement made possible with the election of the Caucus of Rank and File Educators slate in June 2010.
Prior to CORE’s victory, the CTU was run as a top-down union with little member involvement. The new leadership took on the task of forming contract action committees to educate and involve the members, and to develop new rank-and-file leaders, in 615 schools. The union organized three conferences to strategize and implement the contract campaign.
The union put members in motion, with 6,000 attending a rally and march in downtown Chicago four weeks ago. The teachers joined up with the community-labor alliance Stand Up Chicago, which was demonstrating at the Chicago’s Mercantile Exchange shareholder meeting. Unions have been protesting the Merc for months because the trading group sucks up $77 million a year in handouts from the state while Illinois chops budgets for human services.
The exhilaration of the members was palatable: They roared as they marched. One activist told me during the march, “I’ve been a teacher and CTU member for 10 years and this is the proudest I’ve ever felt of my union.”
Following the dictates of the new state law, a three-person fact-finding committee will issue a non-binding report by mid-July, but only on compensation issues. The committee is composed of one representative of the union, one from CPS, and an independent reviewer.
The strike authorization vote gives the union’s House of Delegates—the CTU’s 800-member governing body—the authority to call a strike. CTU’s contract expires June 30, but a strike would not start before the first day of classes for most schools, September 4.
The labor working group of Occupy Chicago has invited labor and community activists to a June 26 meeting to plan a teacher solidarity summer outreach campaign.

Steven Ashby is a labor educator at the University of Illinois, has assisted the union’s contract campaign planning, and observed the four-day ballot count.

Published in: on June 14, 2012 at 8:26 pm  Comments (3)  
Tags: , , ,

Regression to the mean?

It was suggested to me that the fact that very few teachers remained in the 90th percentile for two years in a row in NYC’s value-added madness (my description, not his) is simply yet another case of regression to the mean. That’s a phenomenon where very tall parents tend to have kids that are a bit shorter than they are, and very short parents have children that are taller than them.

Perhaps. But we definitely from ordinary observation that that height, hair color and skin color (but not tattoos or most illnesses) are rather inheritable: tall parents tend to have kids that are taller than most, and short parents have kids that are shorter than most, and so on.

But when the data is a blob showing almost no correlation at all, then regression to the mean doesn’t really mean the same thing. I mean, it starts to become just random variation. Yaknow whatI mean?

OK, let’s look at NYC again.

I just figured out how to get Excel to count some stuff for me in a neat and efficient manner. It counted for me all of the NYC public school teachers who were at or above the 80th percentile in the value-added measurement scheme they'[ve been using there for sy 0506; (That would be considered excellent.) I also had it check to see whether they were also in the 80th percentile during SY 0708, two years later. Or not.

If we trust my programming of Excel, there were exactly 161 such teachers who were in the 80th percentile rank or higher during both years.

But I also had it count how many teachers “dropped”, so to speak. I found there were 545 who were below the 80th percentile the second measured year. Oh, well, that’s nothing – that’s just regression to the mean, you say.

Well, what about those who go BELOW the mean the second year? That’s more than just regression to the mean. After all, the children of congolese pygmies do not get tall enough to play for the NBA. It would take a lot of intermarriage for several generations for that to happen (sorry about that, Bugsy Malone).

In New York city, I found that 146 teachers who were high-flyers in 2005-6 (at or above the 80th percentile) were distincly sub-par, scoring at or below the 50th percentile, two years later.

That’s close to the number of high flyers, nand is about 1/3 of those who “dropped.”

 

My conclusion:

This ‘value added’ stuff is worthless. It has no real predictive value. It doesn’t tell us anything we really want to know, even on its own terms. Plus, it’s measuring the wrong things — but that’s the subject for many more columns to come, and not just by me.

 

Published in: on March 5, 2012 at 10:27 pm  Comments (3)  
Tags: , ,

More Value Added Comparisons

Someone who professes to understand Value-Added scores better than me claims that my graphs for NYC are meaningless because the scores for 2007 were inflated; he claimed that the overall year-to-year and year-to-career value-added correlation coefficients are much higher than what I found — thus, VA is really useful, just not my particular graphs..

Taking this objection seriously, I decided to leave out SY 0607, and compare SY 0506 to SY 0708. Same exact teachers, same exact subjects and grade levels, same exact schools, obviously different (but quite similar) kids.

Here is the scatterplot of what I found. Again, I asked Excel to calculate a line of best fit, and it drew it. Notice that the r-squared correlation value is about 0.05 — seriously LOW. Notice also that this scatterplot is basically a blob again, again a classic example of one variable showing very little correlation with another. (West Virginia’s map has a much more defined shape!) In any case, there are lots (hundreds? thousands?) of teachers with positive VA scores in the first year and negative VA scores the third year, and vice-versa. Only an easily countable handful of teachers have scores of +0.2 or better both years, or worse than -0.2 both years. Out of all of the thousands of teachers. And I bet those are all accidents as well.

So, in other words, I find, as did Gary Rubenstein, that there is extremely little correlation between two things that should be, you would think, very close to a perfect 1.00 correlation. (In the real world, of course, you almost never get a 1.00 correlation between any real entities or quantities. However, when you are talking about the scores of teachers who have been teaching IN THE SAME SCHOOL, THE SAME SUBJECT, THE SAME GRADE LEVEL for three straight years, then you would think that their performances would be rather similar all three years. If anything, they would normally get better unless they had suffered some sort of physically or mentally debilitating injury or illness (often from old age and the incredible amount of stress). In particular, a lot of teachers will admit to you that they absolutely sucked at teaching during their first year, but that they then figured out a lot of those errors and tried not to make the same ones the next year, so they really improved, or else they quit. But these folks didn’t quit. These are at the very least three-year veterans, which in DC would make them eligibility for department or grade level chair at their school as a result of seniority alone, since so many of the older teachers have quit or retired, and the turnover and attrition over the last few years among the newest hires in our school system is probably unprecedented in the history of education. (Perhaps not, but it’s a subject I’d like to pursue.)

 

Finally, while I admit that I exaggerated a bit (for effect) when I said that the shapes of these graphs, and the very low computed values for the r-squared coefficient of linear correlation, made value-added about as predictive as numerology. I thought about that particular exaggeration and wondered how serious it was. So, even though I have participated in a fairly large number of courses on calculating probabilities and distribution, it’s always a bit fraught with error: Have we counted all of the possibilities? Have we left any out? Have we double-counted any of them? Is there a much better, faster, or less error-prone method hidden right around the corner?

 

To make a long story short: the Monte Carlo method is a great way of deciding, say, how likely something is to happen. It’s called “Monte Carlo” because it’s very much like gambling in a casino, except you a4ren’t betting any5thing except your time. You just roll some dice (they might be funny-looking non-cubical polyhedra) or spinning a wheel or throwing darts or spattering paint or vaporized metal… And then you see what happens, and draw conclusions. Today, it’s 4really easy t6o do.

So I decided to see whether, in fact, the number of letters in the teachers’ names had any correlation with their Value Added scores. (I thought it was possible, tho not very likely.) I discovered that Excel found the r-squared constant was about 0.000000. That is zero correlation, my friends. Here is one such scatterplot:

The vertical axis, which goes up the middle, is the number of letter in the teachers’ first name times the number of letters in their last name as listed in the spreadsheet. The horizontal axis, which is at the bottom of the page, is their 2005-2006 value-added score, which can be either negative (theoretically bad) or positive (supposedly good). To me, it sort of looks like bush that hasn’t been pruned in several years – a classic case of no correlation at all.

I asked Excel to draw and calculate the line of best fit. It’s the green, nearly-horizontal line near the center of the graph. Notice the r-squared value: 6E-05, which for all of you innumerates out there, means 0.00006, which is seriously smaller (three orders of magnitude smaller) than 0.05; i.e., one-thousandth as big.

Notice that I’m only using r-squared. Someone objected that i should use just r. If you want, take the square root of all of the correlations I had my computer calculate, and you’ll get r. Compare and contrast.

So, in any case, I definitely did exaggerate.

Now I Understand Why Bill Gates Didn’t Want The Value-Added Data Made Public

It all makes sense now.

At first I was a bit surprised that Bill Gates and Michelle Rhee were opposed to publicizing the value-added data from New York City and other cities.

Could they be experiencing twinges of a bad conscience?

No way.

That’s not it. Nor do these educational Deformers think that value-added mysticism is nonsense. They think it’s wonderful and that teachers’ ability to retain their jobs and earn bonuses or warnings should largely depend on it.

The problem, for them, is that they don’t want the public to see for themselves that it’s a complete and utter crock. Nor to see the little man behind the curtain.

I present evidence of the fallacy of depending on “value-added” measurements in yet another graph — this time using what NYCPS says is the actual value-added scores of all of the many thousands of elementary school teachers for whom they have such value-added scores in the school years that ended in 2006 and in 2007.

I was afraid that by using the percentile ranks as I did in my previous post, I might have exaggerated or distorted how bad “value added” really was.

No worries, mate – it’s even more embarrassing for the educational deformers this way.

In any introductory statistics course, you learn that a graph like the one below is a textbook case of “no correlation”. I had Excel draw a line of best fit anyway, and calculate an r-squared correlation coefficient. Its value? 0.057 — once again, just about as close to zero correlation as you are ever going to find in the real world.

In plain English, what that means is that there is essentially no such thing as a teacher who is consistently wonderful (or awful) on this extremely complicated measurement scheme. How teacher X does one year in “value-added” in no way allows anybody to predict how teacher X will do the next year. They could do much worse, they could do much better, they could do about the same.

Even I find this to be an amazing revelation. What about you?

And to think that I’m not making any of this up. (unlike Michelle Rhee, who loves to invent statistics and “facts”.)

PS:

I neglected to give the links to where you can find the raw data. (Warning: some of these spreadsheets are enormous); Here they are:

http://www.ny1.com/content/top_stories/156599/now-available–2007-2010-nyc-teacher-performance-data#doereports
or, if you prefer a shorter URL, try this one:

http://tinyurl.com/836a8cj

Now Michelle Rhee Needs Special Security Protection?

Check out the email someone forwarded to me from a California lecturing group (MPSF) that has booked Michelle Rhee. Apparently they are afraid of big, bad teachers coming after her.

Here is the website for the lecture series if you want to make their dreams nightmares come true. I think it’s important for the public to realize that MR is an utter charlatan, profiting from destroying public education for our neediest citizens.

Anybody in the DC  area interested in seeing “The Inconvenient Truth Behind Waiting For Superman”, which points out how clearly flawed the pro-Rhee propaganda film “Waiting for Superman”?

If you are, let me know. I’ve got a copy, and more folks should see it.

As usual, it’s easy to leave comments on this blog, but you have to look hard to find the button  for comments: it’s minuscule.

Published in: on February 3, 2012 at 2:37 pm  Comments (2)  
Tags: , , ,
Follow

Get every new post delivered to your Inbox.

Join 395 other followers