Utter, Stunning Failure by Rhee, Kamras, Henderson et al:

“Mr. Teachbad” did such a great job analyzing the utter failure of these contemptible liars that I hope he won’t mind that I re-post it in full:

====================

16 MAR 2013       by teachbad

Well, shit…THAT didn’t work. Now what?

This is stunning.

You remem­ber Michelle Rhee, right? She came to turn the DC pub­lic school sys­tem around. In 2007 she grabbed this city by the throat and shook it into sub­mis­sion.  Teach­ers were fired by the hun­dreds and prin­ci­pals by the dozens. Thou­sands have left the sys­tem because they did not want to work under the con­di­tions Rhee and Jason Kam­ras, her chief teacher tech­ni­cian, were imposing.

That was fine with her. Screw ‘em.  She would find new peo­ple who were will­ing to work hard and believed in chil­dren. Mil­lions upon mil­lions of new dol­lars were found and spent on telling teach­ers how to teach, reward­ing the lap­dogs and fer­ret­ing out the infidels.

Big change never comes easy. You can’t make an omelet with­out break­ing some eggs, etc. But if the right peo­ple have the resources and the courage to make and fol­low through with the tough deci­sions, great things can happen.

After five years, how is DCPS doing? A DC Fis­cal Pol­icy Insti­tute study released ear­lier this week has eval­u­ated the work of Rhee and her suc­ces­sor, Kaya “sucks-to-be-me” Hen­der­son. A write up of the study by Emma Brown can also be found at the Wash­ing­ton Post.

The prin­ci­pal find­ing of the study was that the “share of stu­dents scor­ing at a pro­fi­cient level at the typ­i­cal school fell slightly between 2008 and 2012.”

Whatch­utalk­in­boutwillis? Seri­ously? Read that again. Oh…my…God.

But hold on. That can’t really say every­thing. And what the hell is a “typ­i­cal school”? Let’s dis­ag­gre­gate the data.

Fair enough. The first thing to notice is that pub­lic char­ter schools are doing bet­ter than DCPS schools; not by a huge amount, but it is notice­able and across the board. So there’s that.

More impor­tantly, inter­est­ing pat­terns are revealed when look­ing at schools across these five years by income quin­tiles. Then, as now, the best per­form­ing schools are in the wealth­i­est parts of town and the worst per­form­ing schools are in the poor­est parts of town. That almost goes with­out say­ing. But have schools in the poor­est parts of the city begun to catch up? After all, that’s what this is sup­posed to be all about; clos­ing the achieve­ment gap. How’s that going?

There’s no easy way to say this, so I’ll just come out with it:

      Pro­fi­ciency rates have increased in the four wards with the high­est incomes. Pro­fi­ciency rates have fallen in the four wards with the low­est incomes.   

So, Michelle, Kaya and Jason…it appears you have man­aged to INCREASE the size of the Achieve­ment Gap in Wash­ing­ton, DC. And, Michelle, you are now try­ing to export your great ideas to the entire coun­try? If the three of you don’t feel stu­pid by now, you’re even dumber than I thought. You should all resign. Immediately.

But maybe there’s hope. There is a new plan. Not just any plan, but a strate­gic plan. The study notes that DCPS’s newCap­tial Com­mit­ment plan (yawn) sets the “ambi­tious goal of increas­ing pro­fi­ciency rates at the 40 low­est per­form­ing schools by 40 per­cent­age points by 2017….Given the DC CAS score trends over the past four years, it would appear that DCPS needs to under­take sub­stan­tial changes to the way it oper­ates to make this goal a reality.”

Wait. Didn’t we just do that?

——— Mr. Teach­bad

 

A Few of Diane Ravitch’s Contributions

This is from Richard Hake, who apparently is paraphrasing what David Denby at <http://nyr.kr/RPfOkF> wrote (paraphrasing; supplemented by references to Ravitch’s critiques in “The New York Review of Books”:

Diane Ravitch has emerged as one of the leading opponents of the education-reform movement. She has:

1. Written a series of scathing rebuttals of reform measures in “The New York Review of Books”:
a. “The Myth of Charter Schools” <http://bit.ly/h0Lx8Q&gt;;
b. “School ‘Reform’: A Failing Grade”<http://bit.ly/TclFCY&gt;;
c. “Schools We Can Envy” <http://bit.ly/QqtdTi&gt;;
d. “How, and How Not, to Improve the Schools” <http://bit.ly/RPBDAO&gt;;
e.  “Do Our Public Schools Threaten National Security?” <http://bit.ly/10hxmth&gt;;
f. “In Mitt Romney’s Schoolroom” <http://bit.ly/TcmHxS&gt;; and
g. “Two Visions for Chicago’s Schools” <http://bit.ly/SKjkeA&gt;.

2.  Written some two thousand posts on a blog <http://dianeravitch.net/ > she started in April, which has received almost a million and a half page views.

3. Published “The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education”[Ravitch (2010a)] at <http://amzn.to/pAjeZU&gt;.

4. Barnstormed across the country giving speeches berating the reform movement, which, in addition to test-based “accountability,” also supports school choice and charter schools (public institutions that often receive substantial private funding and are free from many regulations, such as hiring union teachers in states that require it), and which she calls a “privatization” movement. The reform movement has the support of President Obama and his Education Secretary, Arne Duncan; it is also championed by the Republican Party; by many governors, mayors, and schools chancellors; and by a variety of wealthy entrepreneurs and fund managers, including Bill Gates, Mark Zuckerberg, and Whitney Tilson. It has changed educational thinking in states such as Florida, Wisconsin, and Louisiana, and in cities such as Washington, D.C., New York, Los Angeles, and Chicago.

5. Argued that the reform movement is driven by an exaggerated negative critique of the schools, and that it is mistakenly imposing a free-market ethos of competition on an institution that, if it is to function well, requires cooperation, sharing, and mentoring.

Part 3: Enrollment Trends In DC’s Public Schools and Charter Schools

I notice some interesting trends when I look at these three charts from the Washington, DC Office of the State Superintendent of Education, or DC OSSE.

Do you see what I see?

This first one is enrollment by grade grouping for all DC’s publicly-funded schools, that is, both the regular public schools and the charter schools, combined:

The next one is just for the regular DC public schools:

and the last OSSE graph is just for the charter schools, which are publicly funded but privately run:

One thing I note is that in preK-3, and in Adult Education, and in Special Education schools, there are now more students enrolled in the charter schools than in the regular public schools.

(Not so in the other grade strands.)

The Carlos Rosario adult education school is a charter school,  and apparently well-funded, may be part of the reason for the surge in Adult Ed students in the charter realm. It’s located near the home of a family member, and seems to have a lot of students. I am not sure what’s going on with the special education schools (though as I’ve noted before, it’s awfully weird that nearly every single special education student at these schools, whether charter or public, tests either “proficient” or “advanced”, when a substantial portion of them are unable to feed, dress, or bathe themselves, much less read).

I do not know why the overall enrollment for grades 4 and 5 is down for all publicly-funded schools in DC as a whole, and I am not at all sure why the number of DCPS students in those two grades is down by five percent.

I had sort of expected that the small, one-percent rise in regular DCPS population would be mostly from growth in Pre-K. That turned out not to be the case. If you just count “traditional” enrollment in grades Kindergarten through 12, enrollment went from 37,927 to 38,397, which is an increase ofr 470 students, roughly 1.2%; and that’s about the same as the corresponding change in DCPS as a whole, i.e. 1.5%.

In the charter schools, too, the enrollment growth in grades K through 12 is about 11.1%, not really different from the overall charter school growth of 11.0%.

That’s what I see.

What do you see?

Part 2: DCPS and DC charter school enrollments, 2011-2013

Let’s now look at what happened to the enrollment in the ordinary DC public schools, and in the charter schools, separately, over the same period (sy 2001/2 through sy 2012/13), so far.

Here is OSSE’s somewhat misleading graph for the regular DC public school system:

Once again, they are using a vertical scale that doesn’t go to zero, which exaggerates the amount of change. Just looking at the height of the bars, you might think that in 2010, there were only half as many students as in 2002. But that is definitely not the case: it was only a drop of about 32%. That’s serious, but not the same as a 50% drop.

Here is the same data, reformatted by me:

Here, the height of the bars is proportional to the enrollment. And you can see that in the years after 2008, there has been very little change one way or the other in the enrollment of regular DC public schools.

Now let’s compare that with the enrollment in the DC charter schools, which has seen a steady and dramatic increase. For once, the OSSE graph is NOT misleading!!!

The reason that the last blue bar on the right is over three times as tall as the first bar on the left is that, in fact, the charter school population during the current year (2012-13) is in fact nearly three and a half times as much as the charter school population was in 2001-2. That is explosive growth!

Now let’s look at market share, so to speak. What percentage of the students attending publicly-funded pre-K-12 schools in DC are going to regular public schools and what percentage are attending charter schools? The following graph, prepared by me, lets you see just that:

If you follow the trends, it would appear that in a few years, there will be more students in “public” charter schools in DC than in the regular public schools.

Reactions from our public officials? A quote from Mayor Vincent Gray: “One of the strongest indicators that our school system is improving is a steady increase in enrollment numbers – an increase I’m proud to see we have once again achieved,” said Mayor Vincent C. Gray. “This marks the largest enrollment increase in the District’s public schools in 45 years.”

Again, factually correct, but rather an exaggeration. He and his superintendent of schools appear to starve regular DCPS classrooms, enlarge an already bloated and overpaid central office bureaucracy, while funneling cash via foundations to the charter schools. DC now has a larger percentage of its school-age population in charter schools than any other city except for Katrina-battered New Orleans, which mostly means that DCPS central administration remains utterly incompetent at running a school system.

A Look at population trends in DC’s school population

I’m going my best to play it straight.

In the following fiew posts I will try to give you a clear look at how the publicly-funded student enrollment in Washington, DC has been trending over the past decade or so.

I’ll then make a few predictions of how I think things will continue in the next few years.

And then make some judgments on what these records mean.

I think that graphs are often one of the very easiest ways to make things clear, but as Darrell Huff wrote a long time ago in a classic work called “How to Lie With Statistics“, you can still use them in many ways to mislead if you want to.

Here is the very first graph given by DC’s Office of the State Superintendent of Education, or OSSE:

Looks like a huge jump, right? Maybe the latest enrollment is about twice as much as it was at its low point just after Rhee got here to save the day in 2008, right?

No.

It’s not such a huge jump.

And it’s for the regular DC public schools combined with the DC charter schools (privately run tho funded by taxpayers). Not just DCPS, which I’ll look at in a subsequent blog.

The scale is really misleading. The total population is in fact increasing, but (a) most of it seems to have occurred after Rhee left, and (b) from the current high of 80,854 from a nadir of 70,922 is only a 14% increase, or about one-seventh, not a doubling of population.

One’s eye wouldn’t trick one so if one used a scale that went all the way to zero on the vertical axis. (BTW, this is precisely one of Huff’s methods of lying with statistics!) So here is what I think is a fairer way to represent the data, with a scale that goes from 0 to 90,000.

You can see that over the past decade or so, there was a modest drop, followed by a modest rise. These are NOT huge changes, folks!

Is there something special and weird going on? Not really. The population of DC is rebounding somewhat, as well. Take a look at this graph prepared by Google and the US Census Bureau, not by me:

I hope it’s not a surprise that the school enrollment numbers and the total DC population of all ages do not move in lockstep! But, as a general rule, if you get more adults, they have a mysterious way of making babies, and those little’uns eventually do grow up and go to school somewhere!

How Far We Have NOT Come

Before the rise of charter schools in Washington, DC, the claim was that there were too many school buildings. A common complaint was that DC Public School system was too thinly spread and chaotic.

By my count as of 2012, we have at least 189 different schools, a good number of them charter schools, of course. Unless I’m completely mistaken, this is MORE separate schools than we had at any time before-hand. (A few more schools are not counted at all…)

Of all of these schools, only a tiny fraction of them have any white kids at all. I count a total of twenthy-seven of them  that have a tested population consisting of more than FIVE percent non-hispanic whites, in a city whose population is about 35% white and about 51% black, according to the 2000 census.

Here is the list of all of those schools with at least a handful of white kids, in alphabetical order, with their stated percentage of non-hispanic white students. To make the distinction clearer for those not highly familiar with DC schools, I wrote the abbreviation “(ch)” after the names of the charter schools.

1              Brent ES               34%

2              Capital City – Lower (ch)        24%

3              D.C. Bilingual (ch)      23%

4              Deal MS               42%

5              E.L. Haynes – Georgia Ave  (ch)          9%

6              Eaton ES               31%

7              Ellington School of the Arts          9%

8              Elsie Whitlow Stokes Community Freedom (ch)          7%

9              Hardy MS            9%

10           Hearst ES             19%

11           Hyde-Addison ES             39%

12           Janney ES            68%

13           Key ES   69%

14           Lafayette ES       69%

15           Montessori School @ Logan        40%

16           Murch ES             58%

17           Oyster-Adams Bilingual EC           22%

18           School Without Walls HS               35%

19           Stoddert ES        45%

20           Stuart-Hobson Middle School     13%

21           Tuition Grant-DCPS Non Public  15%

22           Two Rivers – Elementary (ch)                40%

23           Two Rivers – Middle (ch)       14%

24           Washington Latin – Middle (ch)           36%

25           Washington Yu Ying (ch)       21%

26           Watkins ES (Capitol Hill Cluster)       19%

27           Wilson, Woodrow HS     19%

I was going to list all the highly-segregated schools, but I realized even if I defined “segregated” as having more than 85% of the school population as being of one single ethnic/racial group or being poor, that that would be an exceedingly long list! In fact, I count exactly 200 instances where a school is highly segregated by ethnicity or poverty or both! (A school like Drew, which is 98% black and 92% poor, gets counted twice.)

We have 28 public and charter schools where 90% of the students are poor, as measured by being on the Free and Reduced Meals list. Here they are:

1              Aiton ES

2              Amidon-Bowen ES

3              Center City – Brightwood (ch)

4              Center City – Congress Heights (ch)

5              Cesar Chavez – Bruce Prep (ch)

6              Community Academy – Amos III (ch)

7              Community Academy – Butler (ch)

8              Community Academy – Rand (ch)

9              Drew ES

10           Excel Academy (ch)

11           Ferebee-Hope ES

12           Friendship – Blow-Pierce (ch)

13           Friendship – Southeast Academy (ch)

14           Hendley ES

15           Howard Road Academy – Main (ch)

16           Imagine Southeast (ch)

17           Kenilworth ES

18           King, M.L ES

19           Kramer MS

20           Malcolm X ES

21           Mary McLeod Bethune – Slowe_Brookland (ch)

22           Maya Angelou – Middle (ch)

23           Moten ES at Wilkinson

24           Patterson ES

25           Savoy ES

26           Stanton ES

27           Tree of Life Community (ch)

28           Tubman ES

 

As my title laments, how far we have NOT come in reaching the dream of which Dr. King spoke.

Double-Digit Increases and Decreases in NCLB Pass Rates: Real or Fraudulent?

A lot of DC public and charter schools have had a lot of double-digit year-to-year changes in their published proficiency rates from 2008 through 2012.

Of course, that sort of change may be entirely innocent, and even praiseworthy if it’s in a positive direction and is the result of better teaching methods. However, we now know that such changes are sometimes not innocent at all and reflect changes in methods of tampering with students’ answer sheets. (And we also know that DC’s Inspector General and the Chancellors of DCPS are determined NOT to look for any wrong-doing that might make their pet theories look bad.)

Whether these are innocent changes, or not, is for others to decide – but these schools’ scores are worth looking at again, one way or another. If it’s fraud, it needs to be stopped. If double-digit increases in DC-CAS pass rates are due to better teaching, then those methods need to be shared widely!

What I did was examine a spreadsheet published by OSSE and Mayor Gray’s office and examine how the percentages of “proficient” students in reading and math at each school changed one year to the next, or from one year to two years later for the period SY 2007-8 through SY 2011-12, five full years. I then counted how many times a school’s listed percentage of “proficient” students went up, or went down, by ten or more percentage points, from one year to the next, or from one year to two years later.

One charter school, D.C. Preparatory Academy PCS – Edgewood Elementary Campus, had ELEVEN double-digit changes from year to year or from one year to two years later. All were upward changes. Perhaps these are really the results of educational improvements, perhaps not. I have no way of knowing. If it’s really the result of better teaching, great! Let their secrets be shared! If it’s not legitimate, then the fraud needs to end.

Two regular DC public elementary schools, Tyler and Hendley, both had TEN double-digit changes measured in the same way. Both had four increases of 10% or more, and both had six decreases by the same amount.

Six schools had NINE double-digit changes. After the names of each school, I will list how many of these were in the positive and negative directions (i.e., up or down). Here they are:

1. Burroughs EC (3 up, 6 down)
2. D.C. Bilingual PCS (8 up, 1 down)
3. Kimball ES (2 up, 7 down)
4. Meridian PCS (5 up, 4 down)
5. Potomac Lighthouse PCS (6 up, 3 down)
6. Wilson J.O. ES (2 up, 7 down)

Thirteen schools had EIGHT double-digit year-to-year changes in proficiency rates. I will list them similarly:

1. Aiton ES (0 up, 8 down)
2. Barnard ES (Lincoln Hill Cluster)  (2 up, 6 down)
3. Cesar Chavez PCS – Capitol Hill Campus (6 up, 2 down)
4. Coolidge SHS (3 up, 5 down)
5. Hospitality PCS (4 up, 4 down)
6. Houston ES (3 up, 5 down)
7. Ludlow-Taylor ES (5 up, 3 down)
8. Noyes ES (1 up, 7 down)
9. Raymond ES (1 up, 7 down)
10. Roots PCS- Kennedy Street Campus (5 up, 3 down)
11. Septima Clark PCS (8 up, 0 down)
12. Thomas ES (4 up, 4 down)
13. Washington Math Science Technology (WMST) PCS (4 up, 4 down)

Eighteen schools had SEVEN double-digit year-to-year changes:

1. Booker T. Washington PCS (4 up, 3 down)
2. Brent ES (7 up, 0 down)
3. Community Academy PCS – Butler Bilingual (7 up, 0 down)
4. Garrison ES (2 up, 5 down)
5. Hearst ES (0 up, 7 down)
6. Imagine Southeast PCS (6 up, 1 down)
7. LaSalle-Backus EC (1 up, 6 down)
8. Leckie ES (2 up,                 5 down)
9. Marie Reed ES (2 up, 5 down)
10. Martin Luther King ES (3 up, 4 down)
11. McKinley Technology HS (7 up, 0 down)
12. Payne ES (5 up, 2 down)
13. Ross ES (6 up, 1 down)
14. Sharpe Health School (4 up, 3 down)
15. Takoma EC (0 up, 7 down)
16. Tree of Life PCS (3 up, 4 down)
17. Turner  ES at Green (3 up, 4 down)
18. Two Rivers Elementary PCS (7 up, 0 down)

 

Seventeen schools had SIX double-digit year-to-year changes in proficiency rates:

1. Bruce-Monroe ES at Park View (2 up, 4 down)
2. Burrville ES (1 up, 5 down)
3. C.W. Harris ES (2 up, 4 down)
4. Center City PCS – Capitol Hill Campus (6 up, 0 down)
5. Center City PCS – Trinidad Campus (5 up, 1 down)
6. Cesar Chavez PCS – Bruce Prep Campus (6 up, 0 down)
7. D.C. Preparatory Academcy PCS – Edgewood Middle Campus (6 up, 0 down)
8. Ferebee Hope ES (1 up, 5 down)
9. Friendship PCS – Blow-Pierce (2 up, 4 down)
10. Friendship PCS – Collegiate (4 up, 2 down)
11. Kenilworth ES (5 up, 1 down)
12. Luke C. Moore Academy HS (4 up, 2 down)
13. Mamie D. Lee School (4 up, 2 down)
14. Roosevelt SHS (3 up, 3 down)
15. Simon ES (3 up, 3 down)
16. Stanton ES (3 up, 3 down)
17. Winston EC (1 up, 5 down)

Let me caution my readers: Just because there are double-digit changes does not in itself mean there is fraud. Student populations can change in average socioeconomic status or composition for all sorts of reasons. Teaching staff and administrators can also change – and so can teaching methodologies, and sometimes entire schools move from one location to another one, with somewhat unpredictable results for good or for the opposite.

However, documented news articles in USA Today and the Atlanta Journal-Constitution, which I have referenced in this blog, have shown convincingly that some of the large swings are definitely due to massive amounts of erasures of incorrect answers, or improper coaching of students during the test by administrators or teachers.

If the increases in pass rates are in fact legitimate, then the rest of the teachers in DC need to know what those secrets are!

In any case, there should be further scrutiny to figure out what is causing such large swings in scores at so many schools.

Note: I got my data here: http://osse.dc.gov/release/mayor-vincent-c-gray-announces-2012-dc-cas-results

A More Realistic Ending to the Movie “Won’t Back Down”

Here is a more realistic version of what would happen in a school that is taken over by a private corporation when parents are duped persuaded to do the “parent trigger” route.

http://jerseyjazzman.blogspot.com/2012/09/wont-back-down-ii-sequel.html?spref=tw

or

More Value Added Comparisons

Someone who professes to understand Value-Added scores better than me claims that my graphs for NYC are meaningless because the scores for 2007 were inflated; he claimed that the overall year-to-year and year-to-career value-added correlation coefficients are much higher than what I found — thus, VA is really useful, just not my particular graphs..

Taking this objection seriously, I decided to leave out SY 0607, and compare SY 0506 to SY 0708. Same exact teachers, same exact subjects and grade levels, same exact schools, obviously different (but quite similar) kids.

Here is the scatterplot of what I found. Again, I asked Excel to calculate a line of best fit, and it drew it. Notice that the r-squared correlation value is about 0.05 — seriously LOW. Notice also that this scatterplot is basically a blob again, again a classic example of one variable showing very little correlation with another. (West Virginia’s map has a much more defined shape!) In any case, there are lots (hundreds? thousands?) of teachers with positive VA scores in the first year and negative VA scores the third year, and vice-versa. Only an easily countable handful of teachers have scores of +0.2 or better both years, or worse than -0.2 both years. Out of all of the thousands of teachers. And I bet those are all accidents as well.

So, in other words, I find, as did Gary Rubenstein, that there is extremely little correlation between two things that should be, you would think, very close to a perfect 1.00 correlation. (In the real world, of course, you almost never get a 1.00 correlation between any real entities or quantities. However, when you are talking about the scores of teachers who have been teaching IN THE SAME SCHOOL, THE SAME SUBJECT, THE SAME GRADE LEVEL for three straight years, then you would think that their performances would be rather similar all three years. If anything, they would normally get better unless they had suffered some sort of physically or mentally debilitating injury or illness (often from old age and the incredible amount of stress). In particular, a lot of teachers will admit to you that they absolutely sucked at teaching during their first year, but that they then figured out a lot of those errors and tried not to make the same ones the next year, so they really improved, or else they quit. But these folks didn’t quit. These are at the very least three-year veterans, which in DC would make them eligibility for department or grade level chair at their school as a result of seniority alone, since so many of the older teachers have quit or retired, and the turnover and attrition over the last few years among the newest hires in our school system is probably unprecedented in the history of education. (Perhaps not, but it’s a subject I’d like to pursue.)

 

Finally, while I admit that I exaggerated a bit (for effect) when I said that the shapes of these graphs, and the very low computed values for the r-squared coefficient of linear correlation, made value-added about as predictive as numerology. I thought about that particular exaggeration and wondered how serious it was. So, even though I have participated in a fairly large number of courses on calculating probabilities and distribution, it’s always a bit fraught with error: Have we counted all of the possibilities? Have we left any out? Have we double-counted any of them? Is there a much better, faster, or less error-prone method hidden right around the corner?

 

To make a long story short: the Monte Carlo method is a great way of deciding, say, how likely something is to happen. It’s called “Monte Carlo” because it’s very much like gambling in a casino, except you a4ren’t betting any5thing except your time. You just roll some dice (they might be funny-looking non-cubical polyhedra) or spinning a wheel or throwing darts or spattering paint or vaporized metal… And then you see what happens, and draw conclusions. Today, it’s 4really easy t6o do.

So I decided to see whether, in fact, the number of letters in the teachers’ names had any correlation with their Value Added scores. (I thought it was possible, tho not very likely.) I discovered that Excel found the r-squared constant was about 0.000000. That is zero correlation, my friends. Here is one such scatterplot:

The vertical axis, which goes up the middle, is the number of letter in the teachers’ first name times the number of letters in their last name as listed in the spreadsheet. The horizontal axis, which is at the bottom of the page, is their 2005-2006 value-added score, which can be either negative (theoretically bad) or positive (supposedly good). To me, it sort of looks like bush that hasn’t been pruned in several years – a classic case of no correlation at all.

I asked Excel to draw and calculate the line of best fit. It’s the green, nearly-horizontal line near the center of the graph. Notice the r-squared value: 6E-05, which for all of you innumerates out there, means 0.00006, which is seriously smaller (three orders of magnitude smaller) than 0.05; i.e., one-thousandth as big.

Notice that I’m only using r-squared. Someone objected that i should use just r. If you want, take the square root of all of the correlations I had my computer calculate, and you’ll get r. Compare and contrast.

So, in any case, I definitely did exaggerate.

Gary Rubenstein is Right: No correlation on Value Added Scores in NYC

One of the things that experimental scientists really should do is to try to replicate each other’s results to see if they are correct or not. I have begun doing that with the value-added scores awarded to teachers in New York City, and I find that I generally agree with the results obtained by Gary Rubenstein.

What I did is looked at the value-added scores, in percentiles, that were “awarded” to thousands of New York City public school teachers in school years 05-06, 06-07, and 07-08. I found that there is essentially no correlation between the scores of the exact same teacher from year to year. The r-squared coefficients are on the order of 0.08 to 0.09 – about as close to random as you can ever get in real life.

Here are my two graphs for the night:

I actually had Excel draw the line of regression, but it’s a joke: an r-squared correlation coefficient of 0.0877 means, as I said, that there is extremely little correlation between what any teacher got in school year 05-06 and what they got in SY 06-07. In the same school. With very similar kids. Teaching the same subject.

And, a similar graph comparing teachers’ scores for school year 06-07 with their scores for 07-08:

So, one year, a teacher might be around the 90th percentile. The next year, she might be around the 10th percentile. Or the other way around. Did the teacher suddenly get stupendously better (or worse)? I doubt it. By the time they are adults, most people are pretty consistent. But not according to this graph. In fact, if somebody is in the 90th to 100th percentile in school year 2006/07, then the probability that they would remain in the same 90th-to-100th-percentile bracket is roughly 1 in 4. If they are in the 0th to 10th percentile in 2006-2007, the chances that they would remain in the same bracket the following year is about 7%!!

What this shows is that using value-added scores to determine if someone should keep their job or get a bonus or a demotion is absolutely insane.