Possible Causes for DC’s Widening Achievement Gap

In my opinion, there are two possible causes. I would like to pose and discuss both of them.

(1) The group of students being tested, scored, and counted in DCPS on the NAEP has changed in nature.

or

(2) There have been major changes in curriculum and/or instructional practices in DCPS.

Let’s consider #1.

We know that this is definitely part of the story,  case, since the NAEP TUDA report (which you can find at http://nationsreportcard.gov/math_2009/math_2009_tudareport/ ) states that for 2009, TUDA no longer counted DC’s charter schools when giving detailed breakdowns of population groups, percentile ranks, and so on for DCPS. (I’ve tried to make this clearer by putting a note to this effect on every graph and table. )  They also point out in their technical notes that if they had also excluded the charter schools in 2007, then the average 8th grade math NAEP score for DCPS in 2007 would have been lower: 244, not 248. However, they do not provide us with enough information for us  to figure out what else would have changed. We are left with a good bit of speculation.

In spring 2009, according to my calculations, there were more 8th graders in regular DC public schools than in the DC charter schools ( 2531 to 1940). However,  higher percentages of students in the 8th grade charter schools scored “proficient” or “advanced” on the DC-CAS  in reading (56% as compared to 40%). So the actual numbers of students making AYP in the public and charter schools in reading in the 8th grade, according to my calculations, were 1,019 in the public and 1,093 in the charter schools – very close. We can infer from the NAEP technical note that in 2007, the 8th grade charter school students had again performed a bit better in math than the regular DCPS students. However, I don’t know off-hand the enrollment figures at that time for the two groups, and I also don’t know if NAEP does some sort of weighted average or not with different students or groups of students. So I really can’t extrapolate to say exactly what the average for regular DCPS students would have been in math for the 8th grade in 2007.

In the 4th grade, the situation is quite different. For one thing, NAEP does not inform us what the true 2007 4th grade regular-DCPS average math score would have been. However, I calculate that there were 3,303 students in the 4th grade  regular DC public schools in April 2009, of which about 48%, or 1,573  students, made AYP in reading on the DC-CAS (not the NAEP). In the charter schools, there were 1,254 students, of whom 492, or about 39%, made AYP in reading. So , unlike in the 8th grade, the 4th grade regular DCPS students did better than the 4th grade charter school students in reading. I think we can safely conclude that in 2007, the regular 4th grade DCPS students probably also did better in math.  If this conclusion is correct, then the regular DC public school students‘ average score in math for 4th grade in 2007 would have been somewhat higher than the 214 that is recorded for year 2007 on page 48 of the NAEP TUDA report.

Unfortunately, I have not done similar detailed breakdown calculations for every single grade level and subgroup at every single public or charter school for the math scores for 2009’s DC-CAS.  I only did them for reading. (And that, alone, took a huge amount of work.) I have done neither math nor reading for 2008, nor 2007, nor any other prior year’s DC-CAS scores.

As you probably know large percentages of students in DC are now attending charter schools, rather than regular public schools. I imagine that every time that DCPS administration does something stupid, then more parents probably decide to move their children to a charter school, hoping that they will do better there. (Despite the statistical evidence to the contrary.) And since there has been so much movement (and with record numbers under Rhee’s administration), the population of DCPS has been generally shrinking.

But why, and how, would this, all by itself, possibly make the achievement gap get larger? Let me try to explain by making up an example that has to do with space rocks.

Suppose a museum has a very large collection of meteorites (rocks from space). These rocks are quite expensive, and they are weighed (and priced) by the gram. One of the curators decides to put into a case a sub-collection, consisting of exactly one 1-gram space rock, one 2-gram space rock, one 3-gram rock, one 4-gram rock, and so on, all the way up to a single  100-gram meteorite. There are 100 rocks in the collection, as you can see below. The average weight of all of these rocks is the sum of all the weights, divided by the number of weights, or 5,050 grams divided by 100 rocks, or 50.5 grams per rock.

I have also shown where the 10th, 25th, 75th, and 90th percentile rocks are, which I will explain later.

Now, if you take out the ten smallest meteorites, the average will increase. Why? Because the total weight of all of the meteorites is now only 4995 grams, and the number of rocks is 90, and when you divide those you get an average of 55.5 grams per rock.  So, dropping the lowest ten rocks increases the average by 5 grams, and the locations of the 10th and 90th percentile rocks have moved as well, as you can see below.

What if the curator had taken out the 10 heaviest meteorites? The sum of all the rocks’ weights would now be 4,095 grams, and when we divide that by 90, we get an average of 45.5 grams per rock. So the average weight has dropped by 5 grams, as you see below, and the locations of the 10th and 90th percentiles have moved, also.

If we take out the 10 middle-sized rocks, that is, the ones from 46 through 55, then you would think that the average weight shouldn’t change. And you would be correct.  The sum of all the rocks’ weights is now 4,545 grams, and when you divide that by 90, you get 50.5 grams again.

But something definitely HAS changed – the middle has disappeared.

You might remember those graphs I made about percentile ranks in DCPS, the nation, and in other large cities. If not, you can look at some prior posts in this blog. Let’s agree that a thing is at the 90th percentile if it is greater than 90 percent of the other things in the group. If we are talking about heights of people, being at the 90th percentile probably doesn’t mean they are 90 inches tall! In fact, men who are about 6 feet, 2 inches are taller than 90% of all other men, so they are at about the 90th percentile.

However, in our example with the 100 original space rocks, the rock that weighs 91 grams has exactly 90 rocks that are lighter than it is, so I will say it’s at the 90th percentile. The rock that weighs 11 grams only has 10 rocks lighter than it is, so let’s agree that it’s at the 1oth percentile. The rock weighing 76 grams is at the 75th percentile, and the rock with a mass of 26 grams is at the 25th percentile.

Now let’s now take out the middle 40 rocks, much like charter schools have taken nearly 40% of the students in DCPS. So the remaining 60 rocks have masses of  1 gram through 30 grams, and then 71 grams through 100 grams. To find the rock that is at the 10th percentile, take 10% of 60. That’s 6. So for a rock to be at the 10th percentile, it only has to outweigh six rocks; those are the ones with masses of 1 g through 6 g. So the rock at the 10th percentile now weighs 7 grams. A rock at the 90th percentile has to have a mass greater than 90% of 60 rocks, which is 54 rocks. I get that  the rock weighing 95 grams is now the one at the 90th percentile. Similarly, I get that the rock weighing 16 grams is now the one at the 25th percentile, and that the rock at the 75th percentile is now the rock with a mass of 86 grams, as you see below.

In the original meteorite collection, as you can see in the first diagram I made, the gap in masses (or weights) between the 10th and 90th percentile was 91 grams minus 11 grams, or 80 grams. The gap between the 25th and 75th percentile is 76 minus 26, or 50 grams.

When we take out the middle 40% of the rocks, the average weight won’t change. But the gaps between the 10th and 90th percentiles HAVE changed: it’s 95 minus 7, or 88 grams, which is 8 grams larger than it used to be. And the gap between the 25th and 75th percentile is now 86  g minus 16 g, or 70 grams, which is 20 grams wider than it used to be.

However, if we just took out the lightest 40% of the rocks, the average mass of the remaining rocks would go up, but the intervals between the 10^th and 90^th percentiles, or etween the 25^th and 75^th percentiles, would get smaller.

My conclusion from all of this is that the charter schools are probably *not* really taking the very highest-performing students from DCPS. Nor are they taking the very worst-performing students from DCPS. I think they are taking the ones that are in the middle of the road, so to speak.

Next time: changes in instructional practices…