The 6-point rise in 4th grade NAEP reading scores is a statistical mirage

A reader who is a LOT more alert than I am (“puzzle palace”) posted this comment, explaining where the 6-point increase in DC NAEP 4th grade reading scores came from. It’s simply because DCPS lost a very large fraction of its lowest-performing students, not because any subgroup (black, white, or hispanic) made any huge gains. I will add some graphs to the post, otherwise I didn’t change a word.

I hope you can follow his/her explanations. It’s marvelous.

==============================================================

“Puzzle Palace” wrote:
The 6 point NAEP 4th grade reading gain is a statistical mirage — purely an artifact of changing demographics.  When broken down by race/ethnicity, white students lost a point since 2007, black students gained 3 and Hispanics gained 1.  Note that these gains are DWARFED by 2005-2007 gains for all groups.

White   Black   Hispanic
2009    257     195     207
2007    258     192     206
2005    252     187     193

(Scores for Asians/Pacific Islanders are not reported.)

[Note: I went back to 1992 in my graph. I used DC at all times, not TUDA, but the differences are very small.]

The puzzle is how could the district average go up 6 points between 2007-2009 if scores for no racial or ethnic did?  Hmm.  That’s a real mystery.

Let’s look at the demographics, percent distribution by race:

White   Black   Hispanic
2009    9       76      13
2007    6       85      7
2005    4       85      9

(Asian/Pacific Islanders make up 1-2 pecent).

Mystery solved.  The apparent 6 point gain is almost entirely a compositional effect of having fewer black students (who score lowest) and more whites and Hispanic (who do better).