More on the Utter Stupidity of NYC’s Value-Added Machinations

I just discovered another weird feature of New York City’s value-added process for teachers.

According to NYC’s own data, a teacher’s percentile ranks for “effectiveness” vary ON AVEARAGE, and for the SAME YEAR, SAME STUDENTS, SAME SUBJECT, SAME CLASS by over 50 percentile ranks.

In other words, a teacher at the median for this variability, about 57 points, could be scoring anywhere from the 20th percentile (very low) on up to the 77th percentile (rather high).

You probably think I’m making this up.

Nope.

Let me give you some raw data and names to chew on.

The following teachers, who are named in the spreadsheets that were obtained by the NYC media, have scores as follows. All of these are NYC PS mathematics teachers. I give you the grade level, followed by what NYCPS says is their lowest value-added percentile rank for 2009/2010, and then their highest possible value-added percentile rank for the same year. In other words, they can’t tell how “good” these teachers really are, even by their own murky methodology.

These are not the exceptional, weird cases. The MEDIAN range of scores for the entire city is 57 points, and if you do a little subtraction, you will notice that in every single one of these cases, their top and bottom scores are 57 percentile points apart.

RHONDA DUFF BAPTISTE              5th Grade            7              64

KRISTIN  DUNBAR                         5th Grade         25           82

DANIELLE DUNNE                        4th Grade            30           87

TONIA EDWARDS                        4th Grade            28           85

ELAINE ELFOND                           8th Grade            36           93

KATHLEEN ESTES MILANO          4th Grade            3              60

STEPHANIE FAIELLA                   5th Grade            8              65

CORDELIA FAULKNER                 5th Grade            7              64

GLORIA FEIERSTEIN                    8th Grade            31           88

SCOTT FLATOW                           4th Grade            29           86

MORGAN FLUSSER                    8th Grade            23           80

DONNA FOSTER                          6th Grade            8              65

PATRICK FOY                             4th Grade            33           90

JENNIFER FRANCKLIN                4th Grade            4              61

ALIZA    FUENTES                       8th Grade            17           74

MARYANN  GANCI                      4th Grade            32           89

(By the way, I don’t know any of these folks, how old they are, what they look like, whether they are strict or lenient, give lots of homework, are tough graders, coach basketball, or anything else about them. But I know that they are real people, real teachers, college grads, and probably a lot like my and my fellow DC teachers except that many of them probably talk funny because they have Noo Yawk accents. Instead of talking normal like y’all do here in DC. ;=) They don’t need to be treated as if their life’s work revolves around a single number — one that nobody seems to be able to pin down very well, at that. Same thing with their students!)

OK, you might be thinking that this only applies to math teachers.

Wrong.

Same deal with English Language Arts teachers, as I show you here below, just like the list up above, which was for math teachers. Again, these teachers have the median (normal) differences between their highest and lowest possible value-added numbers, so they are not exceptional cases. THESE ARE THE TYPICAL CASES.

JULIE BOLAND                               4th Grade            38           95

SHARON BOONE                             4th Grade            3              60

JENNIFER  BRANDES                      8th Grade            0              57

SHARON CANNELLA                       4th Grade            38           95

MONIQUE CARMICHAEL                  4th Grade            40           97

ANATEA CARPENTER                     7th Grade            41           98

REGINA CARROLL                            5th Grade            0          57

CHRISTINA CASSASE                      5th Grade            0           57

CHRISTOPHE CECIL                         5th Grade            39           96

CHEZ DAVIS                                     4th Grade            1              58

JUANITA DAWSON                          4th Grade            6              63

KAREN DOHERTY                           4th Grade            2              59

MICHAEL DONOGHUE                     8th Grade            41           98

JAIME DRAGOON                            4th Grade            2              59

Let me emphasize that these are typical New York teachers. There are OVER ONE THOUSAND, FIVE HUNDRED TEACHERS WHOSE VALUE ADDED SCORES VARY BY 80 PERCENTAGE POINTS OR MORE.

If you’ve forgotten what a percentile rank is, it goes like this: if you are at the 10th percentile for height, that means you are only taller than 10% of your peers, and about 90% of them are taller than you. I.e., you are kinda short. If you at the 86th percentile for height, that means that about 86% of your peers are shorter than you, and you are only shorter than roughly 14% of your peers. In other words, you are rather tall.

If no-one can pinpoint your height any better than by saying you are somewhere between the 41st and 98th percentile, then they haven’t said diddly.

Still don’t believe me? Look at the exact same spreadsheet that I did, I posted it as a google doc at the following URL:

https://docs.google.com/spreadsheet/ccc?key=0AlZJFar_AuNBdGREWTg4NGV0dTlta21IUUNrWXNqTWc

———————–

This range of values is probably their confidence interval, most likely one standard deviation on either side of the theoretical value. However, I don’t see where they actually state that, so I didn’t, either.