More on PIRLS international assessment of nations in reading

It’s utterly amazing:

Twelve minutes after twelve noon on December 12, 2012, (that is, 12:12 on 12/12/12 has come and gone, and somehow the sky still hasn’t fallen.

American fourth-grade students actually did quite well in comparison to about 50 nations in reading and literacy, according to the most recent PIRLS data. Here are two more graphs that I will share with you, which I took from pages 68 and 69 of that report.

This graph is packed full of information!

First of all, notice that the USA is #7 out of all the 40+ nations when ranked by what percentage of students in each country attained scores deemed “advanced”.  We beat nations like Ireland, Israel, NZ, Canada, Taiwan, Denmark, Australia, Germany, Sweden, Netherlands, France, Austria, and many more. That’s not bad.

Further, if you look at the middle, vertical blue line that I drew, you notice that the US has the sixth highest rate in the WORLD of students reaching the “high” benchmark as defined by PIRLS.

Finally, if you look at the median percentages, shown at the very bottom of the page and circled in blue by me, you see that the American rankings are way higher (farther to the right) than any of them.

My next graph shows how a few regions did. I’m going to take this with a bit more salt, however:

 

According to this table, Florida alone is #2 in the entire world. Is that really true? I don’t know; it doesn’t appear to be so highly ranked in reading on the NAEP; it’s tied for 12th place with Delaware, Kentucky, and Montana.

But I think it’s fair to say that American kids aren’t doing as poorly as many pundits and politicians keep saying.

 

 

What About Matthew Ladner’s Closer Look at DCPS NAEP scores?

Ladner says, essentially, that increased numbers of illegally excluded ESL students brought heroic DC chancellor Rhee to grief – that is, when NAEP no longer allowed them to be excluded. Making their scores count made his heroine look bad, he concludes.

His solution? Exclude them all and look what wonders appear! The poor little fourth-graders in DCPS made great strides  – nay, unprecedented, unequalled strides — under the great Michelle Rhee and Kaya Henderson, after all, and despite the nay-sayers.

My response?

Meh.

Despite all his statistical legerdemain – which I have not attempted to replicate, at least not  yet — I still don’t see any miracle occurring in years 2009 and 2011, not even using his own graph and data. What I see is a continuation of past trends. No miracle.

For your edification, I attach both his graph and the original graph. Ladner has clearly learned his lesson from the great “How To Lie With Statistics” — he has stretched the vertical axis to make it appear that the gains were big. He uses about twice as much room going from 180 to 215 as NAEP does in going from 170 to 220.

Look for yourself. And wonder why he didn’t also do this for 4th grade math, 8th grade math, and 8th grade reading.

The top one is from Matthew Ladner at Jay P. Greene’s blog; I added the vertical and horizontal lines.

The bottom one is from the NAEP TUDA site.

 

According to the data produced by Ladner, there was a rise of 27 points in 13 years from 1998 to 2011, a shade more than 2 points per year, or 4 points every 2 years. Some years a bit more (like from 1998 to 2002, when they rose by 12 points in 4 years), some times a bit less. What happened in 2009 and 2011 fits a pure-and-simple straight-line test for all the data quite well.

And, by the way, the folks at NAEP, who are way better statisticians than you, me, or Ladner, I bet, don’t think much of his argument. Although the official NAEP TUDA 4th grade scores for 2011 and 2009 are different from each other by 2 points, they don’t believe that they are significantly different, statistically speaking.

8th Grade TUDA NAEP Reading Results Also Show No Miracles Under Rhee, Henderson et al.

This post concerns the 8th grade NAEP TUDA reading scores for DC and other large cities.  As you may recall, according to Michelle Rhee’s authorized biographer and friend, here is how Rhee claims she could tell if an applicant for a principal position was good enough:

“What’s good to Rhee? If they arrived at their previous school with 20 percent of students reading on grade level and when they left, the number was 70 percent.” (page 132 of The Bee Eater)

Also recall what Rhee claimed to have done in Baltimore: “Over a two-year period, moved students scoring at the 13th percentile on national standardized tests to 90 percent of students scoring at the 90th percentile or higher.”

Keep that standard in mind when you see these graphs and tables.

First, where are DCPS students right now, after more than 4 years of Rhee/Henderson’s “radical reformist” regime? Way down near the bottom of the pack as measured by percentages of students “below basic”, according to NAEP:

But, you say, surely that pitiful figure of 54% of 8th graders reading on a “below basic” level has been getting better under Rhee and Henderson? Guess again, or else look at this graph for DCPS 8th grade reading results over time, and you will see that just the opposite is true, since it’s now worse than in 2002, 2003, 2007, or even 2009:

Now let’s look at those trends in DC again. As you can see, NAEP TUDA 8th grade reading scores are, in fact, slightly LOWER in 2011 than they have been AT ANY TIME while the TUDA study has been going on:

You can see also that while the scores for higher-income kids dropped in 2011, the gap between the richer and poorer students widened by quite a lot in 2009, and is unchanged in 2011 (in both years, it’s 31 points, as opposed to 15 to 21 points during the earlier years).

And let’s look at the gaps between scores of various ethnic groups in DC over time at the 8th grade reading level:

In the graph above, we lack a lot of scores for white students for many years, simply because there were not enough white DCPS students tested at those points in time to satisfy statistical reporting requirements. However, it is clear that the arrival of the protegees of Joel Klein and Michael Bloomberg has NOT benefited any of the 8th grade groups shown: not Hispanic students, not Black students, and not White students.

You might be thinking, “Well, perhaps these gaps are bad in DC, but they are worse elsewhere?”

Pretty much, NO. There are only two cities with larger 8th grade reading achievement gaps between the rich and the poor on the NAEP: Austin and Fresno.

Now let’s look at the gap between scores of white and black 8th graders in various cities, below. As you can see, DC’s gap (circled) is by far the widest such gap: 58 points.

And for Hispanics and whites?

(/sarcasm on) We beat Fresno and Austin! Yippee! DC has the biggest gap! (/sarcasm off)

More on Gaps – This Time, 8th Grade Math NAEP TUDA

This year, the 8th grade math NAEP TUDA results are the only place where the scores in DCPS appear to be going up, but the gap between the top students and bottom students is getting significantly larger.

(Wait until we look at the reading results…)

If you need a little background information, take a look at yesterday’s post for a bit of explanation of what the percentiles mean.

I begin with a graph and table of the gaps in 8th grade math for all US public school students from 2003 to 2011.

As you can see, these scores, which are for all US 8th grade students in public schools, have been slowly but steadily rising since 2003. The gaps between the highest-achieving and lowest-achieving students have been fairly steady or else getting a bit smaller, if you look at the gap between those at the 75th percentile and those at the 25th percentile.

Now let’s look at a similar graph for students in large cities:

While these scores are also rising, I notice one thing that’s different from the previous graph: the kids at the 10th percentile in US large cities score extremely low. That’s the dark blue line at the bottom of the graph, separated from the other lines by a much larger gap than any I’ve noticed before. This phenomenon also shows up because the “90-10 gap” is much larger in this graph than in the previous one.

Now let’s look at the graph and table for Washington DC public schools. Recall that charter school students are excluded from this data starting in 2009.

Quite a different-looking graph, don’t you think? It’s almost like it’s beginning to open up to the right like a fan, because the top teal-colored line, which represents scores of the students at the top of the achievement scale (those at the 90th percentile), and the purple one just below it, are going up faster and faster, leaving the others behind. In 2009, after two years of Rhee, students at the 25th and 10th percentile dropped. To be fair, we don’t know if that drop is real, or is a result of the fact that so many students migrated to charter schools.

We do see that students at the 10th and 25th percentiles had their scores rise significantly in 2011 over 2009, so now they are a couple of points higher than they were in 2007, but that change is not statistically significant.

However, it is clear that overall, the gap in DCPS between the top scorers and the lowest scorers is widening. The 90-10 gap used to be 90 points back in 2003, but has risen to a new high of 107 points in 2011. The 75-25 gap has also reached a new high of 55 points, climbing from 48 points.

Now let’s look at the large city 200 miles to our northeast. I am referring to New York City, of course.

Unlike in DC, where we saw the lines getting farther apart as we move towards the present, in NYC the lines get closer together, which means that the gap between the high-achievers and low-achievers is getting smaller.

But not in a good way.

What is happening in New York’s public schools is that the scores of the students at the top and middle actually dropped over the past 2 years, while the scores of the students at the very bottom (10th percentile) continued to rise. That’s one way to narrow the gap, but it’s the wrong way.

So much for educational miracles happening under undemocratic control of the schools by Mayor Bloomberg! (Remember please that Bloomberg was the person who recommended Michelle Rhee to Adrian Fenty as another miracle-maker.)

Finally, let’s look at Atlanta.

Here we see a pattern that’s different from what we saw elsewhere. The various colored lines seem to be moving just about in lockstep with each other. The gap between the highest and lowest achievers hasn’t changed by much.

—————

Now go back and look at all of those scores.

Have the students in DC miraculously overtaken students in other large cities, or the nation, as essentially promised by the education DEform crowd? No.

Are they on the road to doing so? Heck, no.

Have Rhee, Fenty, Henderson, Bloomberg, Joel Klein, and Vincent Gray delivered any educational miracles in DC or New York? Definitely not.

Gaps between high- and low-achieving students in 4th grade math in DC and elsewhere

Using the NAEP TUDA data, we can see how various groups of students are doing in various cities and here in Washington, DC. In some places, the gap between the highest-achieving students and the lowest-achieving ones is remaining stable. In DCPS, the gap has been getting remarkably wider under the Deformist leadership of Michelle Rhee and Kaya Henderson.

I begin with several graphs and tables that I made, using data from the appendices to the 2011 TUDA report, for 4th grade mathematics over the time period from 2003 to 2011. NAEP gives the average scale score for students who are in the 90th percentile (which is nearly at the top of the charts); those at the 75th percentile (students who score better than about three-fourths of their peers); those at the 50th percentile (kids who are right at the median); those at the 25th percentile (3/4 of the students scoring BETTER than they do); and those at the 10th percentile (only 1 kid in 10 scores LOWER than they do).

We have this data for all 26 TUDA cities, for the nation’s public schools as a whole, and for all large cities added together. And of course for Washington, DC.

I decided to show you also graphs/tables for two other Eastern cities — ones for which we have data going back to 2003: Atlanta and New York.

First, let’s look at national public school data for 4th grade NAEP math scale score, by percentiles:

The top, light blue line represents average scores of kids at the 90th percentile. The bottom, darker blue line represents average scores of students at the 10th percentile.

As you can see, these scores have all been rising very slowly over time, but only by a few points since 2003. The gaps are tabulated in the bottom two rows of the tables by subtracting scores of those in the top ranks minus the scores of the students in the bottom ranks.

The “90-10 gap” means the differences in scores between those at the 90th percentile and those at the 10th percentile. Looking at the bottom two rows of the table, and at the relative distances between the lines in the graph, you can see that the gaps between the various groups have also essentially stayed the same on a national basis.

Now let’s look at a similar graph that’s for all large US cities (places with more than 250,000 population):

Here we see a very similar story, with two differences:

(1) the scores in all US cities seem to be rising faster than the scores in the nation as a whole

(2) the scores in all US cities are lower than those of the nation as a whole.

The gap between the highest-scoring and lowest-scoring students in the nation as a whole hasn’t really changed much over the past 8 years.

——————————————————

Now let’s look at Washington, DC. What differences do you notice?

I don’t know about you, but I notice that the scores for the lowest-scoring students in DCPS seem to be flat or going down over the past few years.

I also notice that the gap between the lowest-scoring and highest-scoring students in DCPS is getting significantly wider. This is not an illusion, but it is troubling.

While the top-scoring kids in DCPS score almost as well as the top-scoring kids nationally or in other large cities, our lowest-achieving students score much lower than their counterparts elsewhere.

—————————————————————-

Is this true in Atlanta and New York? Let’s look at Atlanta first.

In Atlanta, while the scores are generally increasing in all groups, the gap between the top-scorers and the bottom scorers seems to be getting a little wider.  But not nearly so much as in DC under Rhee and Henderson.

(Yes, we know that Atlanta had a major cheating scandal — like here in DC — on their NCLB tests, but this is a different test that is not administered directly by the schools or the teachers. Plus, students in any one classroom who take the test are taking several different forms of the test that gauge student knowledge on entirely different topics. So we can discount entirely the chance that these results are caused by cheating.)

Finally, let’s look at New York:

Isn’t this remarkable? New York has direct mayoral control of the schools, just like DC. Mayor Bloomberg and his current puppet school chancellor have seen scores go DOWN across the board over the past two years as they enact whatever crazy DEforms they are putting into practice. And the gaps between the high and low scorers seem to be getting a bit wider, too.

So, Billionaire-led educvational Deforms cause no more miracles in the Big Apple than they do in the nation’s capital.

Union Membership and Test Scores

Bill Gates, like many other educational Deformers, makes the assumption that having a strong teacher union is a Bad Thing for students, because the union supposedly runs the evil bureaucracy that does its level best to avoid teaching anything to students.

From my viewpoint, having taught for over 30 years in Washington DC, that assumption is so wrong it’s laughable. At best, the union can negotiate pay scales and can represent teachers (good or bad) so that they get some sort of due process when the school administration goes after them. However, much of the time, teachers are too frightened even to speak up, and even with due process, many teachers do get canned or forced out quietly.

In any case, I wanted to look at a particular paragraph where the supposedly highly intelligent Gates compares Florida and Texas, on one hand, with Massachusetts and New York on the other.

Gates makes the assumption that Florida and Texas, which are “right-to-work” states and whose politicians have famously taken a leadership role in the current educational Deform movement, should have students who do better on various standardized tests.

Here’s the quote:

“Asked if the National Education Association and the American Federation of Teachers have any incentive to back school reforms that help kids but also diminish union power, Mr. Gates responds by questioning the scope of that power. “We have heavy union states and heavy right-to-work states, and the educational achievement of K-12 students is not at all predicted by how strong the union rules are,” he says. “If I saw that [right-to-work states like] Texas and Florida were running a great K-12 system, but [heavy union states like] New York and Massachusetts have really messed this up, then I could draw a correlation and say it’s either got to be the union—or the weather.”"

Well, in fact, in general, Texas and Florida generally score LOWER on the NAEP, the only national test we have outside of the SAT and ACT, than do New York and Massachusetts. In fact, students in Massachusetts consistently scores well above all other states in all of the reading and math NAEP tests given in 2009 at the 4th and 8th grade level.

I prepared some graphs using publicly available data comparing 2009 math and reading NAEP scores and percentage of union membership among all employees in each of the 50 states plus DC. I have picked out some of the very highest-scoring states and some of the very lowest-scoring states. (No surprise, when you compare the city of DC versus the states, DC is last.)

Here is a graph showing the 4th grade reading NAEP scores in the y-axis versus the average total percentage of union members, in each state.

Dots that are close to the right-hand edge of the graph represent states with relatively high percentages of workers that belong to labor unions. In each of these graphs, New York has the very highest percentage of union members of all of the states. The state with the very lowest percentage of union members is North Carolina, at about 3.1%, but I didn’t mark it. Can you figure out which one dot represents NC? It’s the dot a bit to the left of the one for Texas.

Dots that are towards the top edge of the graph are the ones with the highest average state-wide 4th grade reading NAEP scale scores, and the ones towards the bottom are the ones with the lowest scores. I labeled the dots for Massachusetts and New Jersey, which are two of the highest-scoring states; they also have pretty high rates of union membership.

The two lowest dots are for Louisiana and the District of Columbia, which also have relatively low percentages of union membership as well.

I drew two red lines. The vertical one represents the median of the percentages of union members among all employees in each state: about 10.8%. So any dot to the right of that line has a higher-than-median union membership level, and conversely for those to the left of that line. The horizontal line represents the median 4th grade scale score for all of the states and DC: 221. Any state scoring above that line has higher-than-median NAEP scale scores for 2009 in reading for the 4th grade, and vice-versa.

When I look at this graph, it almost looks as if higher percentages of union members in a given state seems to be correlated with somewhat higher grades on the NAEP. But the correlation is not very strong.

Here is another graph, this one for 2009 fourth-grade math NAEP scale scores versus union membership in 2009.

Once again, the two lowest-scoring states are ones with low-to-middling percentages of union members: DC and Alabama. The two states with the highest scores are those with high-to-middling percentages of union members: Massachusetts and New Hampshire. NY, FL and TX have middling scores, but are mostly differentiated by their very different percentages of union members. (Frankly, I am not sure how teachers in rural Texas can pay their rent or mortgages without second jobs…)

Here’s another graph, for 8th grade NAEP reading against percentages of union membership in 2009:

Once again, you see that Bill Gates’s assumption that unions are bad for students doesn’t appear to hold water. Both Florida and Texas, states where very few workers belong to labor unions, scored below the median on the 8th grade reading NAEP. Once again, Massachusetts and New Jersey, with higher-than-median memberships in labor unions, scored at or near the top. New York scored about the same as Florida. The bottom scoring states were DC and Mississippi.

Last graph, for 8th grade 2009 math NAEP scores versus union membership:

Again, if anything, the assumption that he made, that union membership is bad for students, is shattered.

Source for union membership is here.

For the various NAEP scores, you have to go to the main data exploration page for the National Assessment of Educational Progress and look around.

More DC NAEP Data

As you can see, the general trends on NAEP scores have been upwards for about 15 to 20 years. So whatever Rhee and her acolyte,  Whitmire may claim, she didn’t have much to do with that. It’s been lots of English, reading, and math teachers, and their students, and their parents, along with improved curriculum and textbooks, that have made the difference. Not a lying SuperWoman.

Real Data on NAEP scores in DC over the past 20 years

I would like to remind you that Michelle Rhee’s and Richard Whitmire’s claims that she single-handedly boosted the NAEP scores in DCPS, way in excess of anything that happened previously, are bogus.

Michelle Rhee and Richard Whitmire aren’t left with very much to brag about concerning her supposed accomplishments in DCPS.

• The ‘Capital Gains’ project was a total failure – even according to the data provided by Harvard professor/entrepreneur Roland Fryer (who has also run a massive randomized experiment in NYC showing that paying teachers money for higher student test scores was counterproductive)
• Bringing in an outside agency to run Dunbar SHS was a total failure
• The situation with the least-trained, least-experienced teachers staffing most of the roughest, lowest-income schools in DCPS is worse now than ever
• The situation with special education is the usual mess
• There are more central-office employees than ever, and they make more money than ever
• It’s clear that the enormous sums she spent on consultants haven’t done a damned thing
• The teachers are totally demoralized
• Real teaching has been replaced by 100% test-prep, all the time
• The method that her favorite principals employed to raise test scores involved simple cheating: long hours of meticulous erasing of wrong answers, after the students and teachers had left the buildings

So let’s look at a few graphs of how NAEP scores have changed over the past 20 or so years, so you can see for yourself whether the 3 years of Rhee made much of a difference.

First, please look at a graph of the average 4th grade DCPS ‘Scale Scores’ on the Math NAEP:

As you can see, they have been rising more-or-less steadily since about 1996.

Next, look at a graph showing the percentage of students “Basic” or above (which really does mean ON GRADE LEVEL)  and “Proficient” or above (which really does mean ABOVE GRADE LEVEL) in math since about 1992. Again, you can see that these percentages have been pretty steadily rising since about 1996. Once again, SuperWoman’s presence apparently didn’t make a huge change.

Next, let’s look at the average scale score for DCPS 8th graders on the NAEP reading test:

Here the record doesn’t go back as far, but besides the drop in 2003 and 2005, the overall trend has been upwards.

I’ll post a few more graphs tomorrow. Right now, I am too tired to keep going.

Another Credibility Gap? Looks Like Rhee, Fenty, and Reinoso are Gaming the Tests to Try to Look Good

Unfortunately, in the field of education, crime does sometimes pay – at least for some people.

In a number of states, local politicians have tried to make themselves and their pet educational De-Forms look good by gaming the testing system. For example, Rod Paige’s educational “miracle” in Houston, TX – the very model for NCLB – was later shown to be based on pushing out low-scoring students, which makes the scores for everybody else in the system look much better. (And helped get GWBush elected President and Rod Paige to become Sec’y of Education.) Other states have made the cut-off scores and tests for NCLB a whole lot easier. That way, even if students aren’t really doing better, they will APPEAR to the public to be doing better.

An alert reader has sent in some data that suggests that sort of “gaming the system” might be happening here in DC, but not in Massachusetts.  (It’s in a comment on a previous blog.) I wanted to make sure the data was correct before I posted the data here as a somewhat more-legible table, so I double-checked the sources and found that it was all correct.

Then, on a hunch, I went back a few years to see if the problem is getting worse here in DC, and how it compares to Massachusetts.

Result: Yes. There is a greater and greater gap between the NAEP and local DC-CAS scores, but that is NOT happening in the Bay State.

Here are the tables I made:

Notice a few things:

In 2007, in DC, the locally-commissioned DC-CAS test results, in both reading and math, and at both the 8th grade and 4th grade levels said that about 15 to 25%  more students were proficient than the national NAEP scores said.

In 2009, the discrepancies between the local DC test and the national NAEP tests were much larger than in 2007: our local test says that roughly 30% more of our students are proficient than the national test does, in all 4 sub-groups (reading, math, 4th grade, and 8th grade). In other words, the DC-CAS seems to be giving a more and more inflated view of how our students are performing.

In Massachusetts, the trend seems to be the other direction. In 2007, the gap between the state MCAS test and the NAEP ranged anywhere from positive 1% to negative 32%. However, 2 years later, the gap went from positive 9% to negative 5%.  So, in other words, the MCAS appears to be improving its match to the NAEP.

And let me emphasize something: in Massachusetts, the MCAS last year gave LOWER scores than the NAEP in half of the cases I sampled.

You can find the Massachusetts results at
http://www.doe.mass.edu/mcas/results.html
. (Notice that they make up new tests every year.)

The NAEP results can be found at
http://nces.ed.gov/nationsreportcard/states/
(but be prepared to dig a bit).

And Fenty, Rhee, and Reinoso bragged about the DC-CAS at http://dcps.dc.gov/DCPS/About+DCPS/Press+Releases+and+Announcements/Press+Releases/Fenty,+Rhee+and+Reinoso+Announce+DCPS+2009+DC+CAS+Scores or here.

——————————————————————

There are all sorts of easy ways of doing that sort of score inflation. One method, which works quite well if you use the same test over and over, is simply to have administrators or teachers cheat.  Blatantly.  And get more proficient at it. (We have all heard allegations of this.)

Another, more subtle, way to inflate the scores is to slightly modify the test from year to year so that it uses slightly easier questions.

Another way is to change the cut-off scores so that whereas it might have required, say, 60% to be “proficient” one year, the next year, you only need to get 45%  of them right. (It has been shown that you can get a passing score on certain high-stakes tests in New York by guessing randomly.)

Or, you simply make it so that the entire curriculum IS teaching to the test – which is what Chancellor Rhee apparently thinks is the proper way to teach.

Possible Causes for DC’s Widening Achievement Gap

In my opinion, there are two possible causes. I would like to pose and discuss both of them.

(1) The group of students being tested, scored, and counted in DCPS on the NAEP has changed in nature.

or

(2) There have been major changes in curriculum and/or instructional practices in DCPS.

Let’s consider #1.

We know that this is definitely part of the story,  case, since the NAEP TUDA report (which you can find at 
http://nationsreportcard.gov/math_2009/math_2009_tudareport/
) states that for 2009, TUDA no longer counted DC’s charter schools when giving detailed breakdowns of population groups, percentile ranks, and so on for DCPS. (I’ve tried to make this clearer by putting a note to this effect on every graph and table. )  They also point out in their technical notes that if they had also excluded the charter schools in 2007, then the average 8th grade math NAEP score for DCPS in 2007 would have been lower: 244, not 248. However, they do not provide us with enough information for us  to figure out what else would have changed. We are left with a good bit of speculation.

In spring 2009, according to my calculations, there were more 8th graders in regular DC public schools than in the DC charter schools ( 2531 to 1940). However,  higher percentages of students in the 8th grade charter schools scored “proficient” or “advanced” on the DC-CAS  in reading (56% as compared to 40%). So the actual numbers of students making AYP in the public and charter schools in reading in the 8th grade, according to my calculations, were 1,019 in the public and 1,093 in the charter schools – very close. We can infer from the NAEP technical note that in 2007, the 8th grade charter school students had again performed a bit better in math than the regular DCPS students. However, I don’t know off-hand the enrollment figures at that time for the two groups, and I also don’t know if NAEP does some sort of weighted average or not with different students or groups of students. So I really can’t extrapolate to say exactly what the average for regular DCPS students would have been in math for the 8th grade in 2007.

In the 4th grade, the situation is quite different. For one thing, NAEP does not inform us what the true 2007 4th grade regular-DCPS average math score would have been. However, I calculate that there were 3,303 students in the 4th grade  regular DC public schools in April 2009, of which about 48%, or 1,573  students, made AYP in reading on the DC-CAS (not the NAEP). In the charter schools, there were 1,254 students, of whom 492, or about 39%, made AYP in reading. So , unlike in the 8th grade, the 4th grade regular DCPS students did better than the 4th grade charter school students in reading. I think we can safely conclude that in 2007, the regular 4th grade DCPS students probably also did better in math.  If this conclusion is correct, then the regular DC public school students‘ average score in math for 4th grade in 2007 would have been somewhat higher than the 214 that is recorded for year 2007 on page 48 of the NAEP TUDA report.

Unfortunately, I have not done similar detailed breakdown calculations for every single grade level and subgroup at every single public or charter school for the math scores for 2009′s DC-CAS.  I only did them for reading. (And that, alone, took a huge amount of work.) I have done neither math nor reading for 2008, nor 2007, nor any other prior year’s DC-CAS scores.

As you probably know large percentages of students in DC are now attending charter schools, rather than regular public schools. I imagine that every time that DCPS administration does something stupid, then more parents probably decide to move their children to a charter school, hoping that they will do better there. (Despite the statistical evidence to the contrary.) And since there has been so much movement (and with record numbers under Rhee’s administration), the population of DCPS has been generally shrinking.

But why, and how, would this, all by itself, possibly make the achievement gap get larger? Let me try to explain by making up an example that has to do with space rocks.

Suppose a museum has a very large collection of meteorites (rocks from space). These rocks are quite expensive, and they are weighed (and priced) by the gram. One of the curators decides to put into a case a sub-collection, consisting of exactly one 1-gram space rock, one 2-gram space rock, one 3-gram rock, one 4-gram rock, and so on, all the way up to a single  100-gram meteorite. There are 100 rocks in the collection, as you can see below. The average weight of all of these rocks is the sum of all the weights, divided by the number of weights, or 5,050 grams divided by 100 rocks, or 50.5 grams per rock.

I have also shown where the 10th, 25th, 75th, and 90th percentile rocks are, which I will explain later.

Now, if you take out the ten smallest meteorites, the average will increase. Why? Because the total weight of all of the meteorites is now only 4995 grams, and the number of rocks is 90, and when you divide those you get an average of 55.5 grams per rock.  So, dropping the lowest ten rocks increases the average by 5 grams, and the locations of the 10th and 90th percentile rocks have moved as well, as you can see below.

What if the curator had taken out the 10 heaviest meteorites? The sum of all the rocks’ weights would now be 4,095 grams, and when we divide that by 90, we get an average of 45.5 grams per rock. So the average weight has dropped by 5 grams, as you see below, and the locations of the 10th and 90th percentiles have moved, also.

If we take out the 10 middle-sized rocks, that is, the ones from 46 through 55, then you would think that the average weight shouldn’t change. And you would be correct.  The sum of all the rocks’ weights is now 4,545 grams, and when you divide that by 90, you get 50.5 grams again.

But something definitely HAS changed – the middle has disappeared.

You might remember those graphs I made about percentile ranks in DCPS, the nation, and in other large cities. If not, you can look at some prior posts in this blog. Let’s agree that a thing is at the 90th percentile if it is greater than 90 percent of the other things in the group. If we are talking about heights of people, being at the 90th percentile probably doesn’t mean they are 90 inches tall! In fact, men who are about 6 feet, 2 inches are taller than 90% of all other men, so they are at about the 90th percentile.

However, in our example with the 100 original space rocks, the rock that weighs 91 grams has exactly 90 rocks that are lighter than it is, so I will say it’s at the 90th percentile. The rock that weighs 11 grams only has 10 rocks lighter than it is, so let’s agree that it’s at the 1oth percentile. The rock weighing 76 grams is at the 75th percentile, and the rock with a mass of 26 grams is at the 25th percentile.

Now let’s now take out the middle 40 rocks, much like charter schools have taken nearly 40% of the students in DCPS. So the remaining 60 rocks have masses of  1 gram through 30 grams, and then 71 grams through 100 grams. To find the rock that is at the 10th percentile, take 10% of 60. That’s 6. So for a rock to be at the 10th percentile, it only has to outweigh six rocks; those are the ones with masses of 1 g through 6 g. So the rock at the 10th percentile now weighs 7 grams. A rock at the 90th percentile has to have a mass greater than 90% of 60 rocks, which is 54 rocks. I get that  the rock weighing 95 grams is now the one at the 90th percentile. Similarly, I get that the rock weighing 16 grams is now the one at the 25th percentile, and that the rock at the 75th percentile is now the rock with a mass of 86 grams, as you see below.

In the original meteorite collection, as you can see in the first diagram I made, the gap in masses (or weights) between the 10th and 90th percentile was 91 grams minus 11 grams, or 80 grams. The gap between the 25th and 75th percentile is 76 minus 26, or 50 grams.

When we take out the middle 40% of the rocks, the average weight won’t change. But the gaps between the 10th and 90th percentiles HAVE changed: it’s 95 minus 7, or 88 grams, which is 8 grams larger than it used to be. And the gap between the 25th and 75th percentile is now 86  g minus 16 g, or 70 grams, which is 20 grams wider than it used to be.

However, if we just took out the lightest 40% of the rocks, the average mass of the remaining rocks would go up, but the intervals between the 10^th and 90^th percentiles, or etween the 25^th and 75^th percentiles, would get smaller.

My conclusion from all of this is that the charter schools are probably *not* really taking the very highest-performing students from DCPS. Nor are they taking the very worst-performing students from DCPS. I think they are taking the ones that are in the middle of the road, so to speak.

Next time: changes in instructional practices…