Sometime this month, the Mayor of DC and the Chancellor of the DC Public Schools will make some sort of announcement on how DC public and charter school students did on the DC-CAS (Comprehensive Assessment System) – the test required by Federal law to be given to every single kid in grades 3 through 8 and in grade 10.
I don’t have a crystal ball, and I haven’t developed any sources willing to risk their jobs by leaking the results to me in advance, but I can make a few predictions:
1. If the results look bad, they will be released right before a holiday or a weekend (a basic public-relations tactic that all public officials learn).
2. If the scores as a whole look good, or if there is some part of the trends that look good, that will be highlighted heavily.
3. There won’t be much of a correlation between the trends on the DC-CAS scores and the National Assessment of Ednucational Progress, which has been measuring student achievement in grades 4 and 8 in reading and math since the 1970s by giving a carefully-selected sample of students in DC and across the nation a variety of different test items in math, reading, and a number of other areas.
4. Even though the DC-CAS results won’t be released to the public for a couple more weeks, clearly DCPS officials and Mathematica staff already have them; they have been firing teachers and principals and “adjusting” – with the benefit of hindsight – the rest of their evaluations to fit the DC-CAS scores and the magic secret formula called “Value Added
You may ask, how can GFBrandenburg predict not much of a match between the DC-CAS and the NAEP?
By looking at the track record, which I will share with you.
I present the average scores of all DC students on both the DC-CAS and on the NAEP over the past quarter-century. The NAEP scores for the District of Columbia have either been pretty steady or have been rising slightly.
As far as I can tell, the statisticians at the National Center for Educational Statistics (NCES) who design, administer, and score the NAEP do a fine job of
A. making sure that there is no cheating by either students or adults,
B. making up good questions that measure important topics, and
C. gathering, collating, and reporting the data in an honest manner.
On the DC -CAS, however, we have had many documented cases of cheating (see point A), I have shown that many of the questions are ridiculous and don’t measure what we teachers were supposed to be teaching (see point B), and I hope to show you that whatever they are doing with the scores does not seem to be trustworthy.
Exhibit number one is a graph where I plot the average scale scores of the students in Washington DC on both the NAEP and on the DC-CAS for fourth grade math:
Allow me to explain.
The bottom blue curve is what DC’s fourth-graders average scale scores were on the NAEP starting in 1992 and going on through 2013. As you can see, since 1996, there has been what appears like more-or-less steady improvement.
(It is very hard, in fact, to see much of a difference in trends before mayoral control over the DC schools and after that time. I drew a vertical black line to separate the ‘Pre-Rhee” era from the “Post-Rhee” era, since Michelle Rhee was the very first Chancellor installed in the DC schools, after the annual tests were given in 2007.)
(As noted, the NAEP scale scores go from 0 to 500, but the DC-CAS scores go from 0 to 100. I decided that the easiest way to have them both fit on the same graph would simply be to divide the NAEP scores by 5. The actual reported NAEP scores are in the little table, if you want to examine them for yourself. You can double-check my numbers by looking around at the NAEP and DC OSSE websites — which are unfortunately not easy to navigate, so good luck, and be persistent! You will also find that some years have two different scores reported, which is why I put those double asterisks at a couple of places on those curves.)
But here’s what’s really suspicious: the DC-CAS scores, shown in red, seem to jump around wildly and appear to show tremendous progress overall but also utterly un-heralded drops.
Which is it?
Slow, steady progress since 1996, or an amazing jump as soon as Wonder Woman Rhee comes on the scene?
In my opinion, I’d much rather trust the feds on this. We know that there has been all sorts of hanky-panky with the DC-CAS, as repeatedly documented in many places. I know for a fact that we math teachers have been changing the ways that we teach, to be more in line with the 1989 NCTM standards and the ways that math is tested on the NAEP. It’s also the case that there has been significant gentrification in DC, with the proportion of white kids with highly educated parents rising fairly steadily.
Slow improvement in math scores, going back a quarter of a century, makes sense.
Wild jumps don’t seem reasonable to me at all.
On the contrary, besides the known mass cheating episodes, it almost seems like DC education officials get together with McGraw-Hill CTB, which manufactures the DC-CAS, and decide how they want to get the scores to come out. THEN they decide which questions to count and which ones NOT to count, and what the cut-off scores will be for ‘advanced’, ‘proficient’ and so on.
Next time: 8th grade math; and 4th and 8th grade reading.
Links to my other articles on this:
Part One (fourth grade math)— this one right here
Part Two (8th grade math)
Part Three (all reading)