Past Performance (on DC’s NAEP and its own CAS tests) Can Give Insight Into Future Performance

Sometimes, looking at the past gives you lots of clues about the future.

From looking at past evidence about the scores of Washington, DC students on both the National Assessment of Educational Progress (NAEP) and on its locally-mandated test (currently known as DC-CAS, or Comprehensive Assessment System), we can make some predictions about the importance (or lack of it) of the results for the 2014 CAS, which were administered about three months ago.

My prediction: the CAS results don’t really matter, one way or the other, because they are so volatile and do not correlate very well with much-more-reliable NAEP. Besides the fact that there have been many documented cases of cheating by adults on the DC-CAS for their own personal gain, it almost seems like the DC-CAS is designed to be manipulated for political and ideological gain, for two main reasons:

(1) In some subjects the DC-CAS has displayed enormous year-to-year score increases that are not at all reflected in the cheat-proof NAEP.

(2) This is despite the fact that between 34% and 42% of all questions are non-scored “anchor” questions designed simply to see if the test gives consistent results from year to year, according to testimony of Emily Durso last September before David Catania’s DC-Council subcommittee on education. Those question’s aren’t scored!

Think about that: something like a third to three-sevenths of all the questions a student is forced to answer isn’t even used to grade the students or teachers. It’s used to help out the testing company.

Which produces unreliable results anyway.

(And even though CTB-McGraw Hill has quite a variety of ways of using statistics to detect cheating by kids or adults, DCPS won’t pay for them to use those methods, I was told by CTB’s head econometrician.)

As promised, here is some of the evidence.

First, we have the DC-CAS (red) and NAEP (blue) scores over the past quarter-century in DC, for 8th grade math students:

Once again, the NAEP scores (in blue) for DC’s 8th graders since 1990 seem to show more-or-less steady growth, especially since the year 2000.

(Keep in mind that in order to plot the NAEP and DC-CAS scores on the same grid, and since the NAEP scores go from 0 to 500 while the DC-CAS scores go from 0 to 100, I divided the NAEP scores by 5.)

The DC-CAS was first administered in 2006, so there are no records from before that year. Note that there are two scores given for the year 2009 (46.0 and 49.8), which are pretty far apart; I can only guess as to why they contradict each other. The two sources are here and here. For the NAEP, the reason for the slightly different scores is quite straightforward: one version of the test allowed accommodations for disabilities, and the other did not.

In any case, up until 2011, the gains by successive cohorts of 8th graders on the DC-CAS math test — whom I often taught when I was a teacher in DCPS — was fairly spectacular, but not at all mirrored by the slow, steady progress shown on the NAEP. And since then, those scores have been fairly flat, especially when we remember that the CAS is scored from 0 to 100 (or, to be technical, from 800 to 899; the fourth-grade test scores go from 400 to 499, and the 10th grade tests are scored from 900 to 999 (probably because they didn’t want to add a fourth digit).

======

Here are the links to the other parts of this article:

Part One  (fourth grade math)

Part Two (8th grade math) — this one right here

Part Three (all reading)