Even the Chancellor Calls the Results ‘Sobering’

The Washington City Paper has an article on the PARCC results with way more graphs and charts than I do, and they quote even Chancellor Kaya Henderson as saying the results were ‘sobering’.

Please remind me why she still has a job?

She and several other speakers said that the PARCC results were more ‘honest’ than the old DC-CAS results, probably because the new ‘passing’ scores are lower than the old ones. I guess that means that it’s more ‘honest’ to say that students are doing worse than we were previously led to believe, under the current regime of all-testing-all-the-time and turn-half-the-students-over-to-unregulated charters?

 

PARCC Results Released in DC

I just got back from watching the public release of the results of the PARCC test that students in Washington DC took about 7 months ago.

(Let that sink in: it took the testing company, and their consultants, and the city’s consultants, over HALF A YEAR to massage the data into a releasable form. So much for having these tests be able to be used to ‘inform instruction’ or help teachers figure out what kind of help their students need. It’s now the last day of November, and the students have been in school since August. What kind of help is that to teachers or parents? And tho I haven’t looked at the released school scores or samples of what the teachers will see, I’m not optimistic. If the past is any guide, the scores themselves will be essentially useless as well…)

(It won’t take so long next time, we were assured…)

I got to see Mayor Bowser, Councilman Grosso, Chancellor Kaya Henderson, [powerless] Superintendent Hanseul Kang, and Deputy Mayor for Education Jenny Niles, and charter honcho Scott Pearson perform and answer some mostly-lame questions from some members of the media.

What we saw were that advanced students in DC (largely white ones) do exceedingly well on this PARCC battery of tests, and that others (blacks; hispanics; SPEDs; students on free or reduced lunch; ELLs; or Students At Risk) do much worse. Which of course is  the very same result we’ve seen on the NAEP for a couple of decades.

In fact, of all the cities and states measured on the NAEP, Washington DC has the very widest gaps in test scores between the Upper Caucasia Haves and the Have-Nots everywhere else, and those gaps are if anything getting wider.

It was interesting to hear Henderson’s defenses of the results, which still showed very low percentages of most students “passing” the PARCC. She said, among other things, that

(1) since students at the lower grades generally scored higher than those at the upper grades, that show’s we are on the right path [seems to me it shows the exact opposite; the longer that students have been exposed to “Reform”, the worse they do… and

(2) It takes a long time, you can’t just expect to turn a switch and have everything be wonderful overnight, we need lots of wrap-around services and a longer school day and school year and support for teachers.

Regarding the latter excuse: isn’t that exactly what teachers were condemned for saying under Chancellor Rhee, whose understudy was none other than Kaya Henderson? Didn’t Rhee imply that the only reason that poor students did poorly in school was that their greedy, lazy teachers, empowered by their evil union, refused to teach them anything? And that anybody who said that it’s a lot harder to teach impoverished students of color with chaotic families (if any) than it is to teach middle-class children with educated parents – why those people were just making excuses for poverty?

 

Where DC is #1 on the NAEP

Of all the states and territories tested on the 2015 NAEP, there is one place where DC is Number ONE!

Unfortunately, it’s not a good #1.

We have, by far, the largest gaps between percentages of white and black students who are deemed ‘proficient’ or better. On every single test (8th grade, 4th grade, reading and math).

DC also the largest gaps between percentages of white and hispanic students – on every single test.

Our DC gaps are at least double the national gaps. And that’s not good. In fact, the gaps are anywhere between double and two-and-a-half times as large as the gaps nationally or the median of all states, as you see here:

gaps b-w and w-h

Kaya Henderson and Michelle Rhee really have some tremendous accomplishments, don’t they?

————

These scores, by the way, are for a carefully-selected sample of ALL students in Washington, DC – public, charters, private, and parochial. Rhee and Henderson and the various DC mayors have been in total control of all public and charter schools since 2007, with a school board that has exactly zero power and a teachers’ union that has lost almost any power to do anything meaningful to support teachers. And we have a teaching and supervisory force that is either brand-new (hired by Rhee or Henderson or by the heads of the many charter schools) or has passed all of the extremely difficult evaluations not once, but many times.

My Predictions for the 2014 DC-CAS Scores

Sometime this month, the Mayor of DC and the Chancellor of the DC Public Schools will make some sort of announcement on how DC public and charter school students did on the DC-CAS (Comprehensive Assessment System) – the test required by Federal law to be given to every single kid in grades 3 through 8 and in grade 10.

I don’t have a crystal ball, and I haven’t developed any sources willing to risk their jobs by leaking the results to me in advance, but I can make a few predictions:

1. If the results look bad, they will be released right before a holiday or a weekend (a basic public-relations tactic that all public officials learn).

2. If the scores as a whole look good, or if there is some part of the trends that look good, that will be highlighted heavily.

3. There won’t be much of a correlation between the trends on the DC-CAS scores and the National Assessment of Ednucational Progress, which has been measuring student achievement in grades 4 and 8 in reading and math since the 1970s by giving a carefully-selected sample of students in DC and across the nation a variety of different test items in math, reading, and a number of other areas.

4. Even though the DC-CAS results won’t be released to the public for a couple more weeks, clearly DCPS officials and Mathematica staff already have them; they have been firing teachers and principals and “adjusting” – with the benefit of hindsight – the rest of their evaluations to fit the DC-CAS scores and the magic secret formula called “Value Added Magic Measurement”.

You may ask, how can GFBrandenburg predict not much of a match between the DC-CAS and the NAEP?

By looking at the track record, which I will share with you.

I present the average scores of all DC students on both the DC-CAS and on the NAEP over the past quarter-century. The NAEP scores for the District of Columbia have either been pretty steady or have been rising slightly.

As far as I can tell, the statisticians at the National Center for Educational Statistics (NCES) who design, administer, and score the NAEP do a fine job of

A. making sure that there is no cheating by either students or adults,

B.  making up good questions that measure important topics, and

C. gathering, collating, and reporting the data in an honest manner.

On the DC -CAS, however, we have had many documented cases of cheating (see point A), I have shown that many of the questions are ridiculous and don’t measure what we teachers were supposed to be teaching (see point B), and I hope to show you that whatever they are doing with the scores does not seem to be trustworthy.

Exhibit number one is a graph where I plot the average scale scores of the students in Washington DC on both the NAEP and on the DC-CAS for fourth grade math:

naep + dccas 4th grade math comparison

Allow me to explain.

The bottom blue curve is what DC’s fourth-graders average scale scores were on the NAEP starting in 1992 and going on through 2013. As you can see, since 1996, there has been what appears like more-or-less steady improvement.

(It is very hard, in fact, to see much of a difference in trends before mayoral control over the DC schools and after that time. I drew a vertical black line to separate the ‘Pre-Rhee” era from the “Post-Rhee” era, since Michelle Rhee was the very first Chancellor installed in the DC schools, after the annual tests were given in 2007.)

(As noted,  the NAEP scale scores go from 0 to 500, but the DC-CAS scores go from 0 to 100. I decided that the easiest way to have them both fit on the same graph would simply be to divide the NAEP scores by 5. The actual reported NAEP scores are in the little table, if you want to examine them for yourself. You can double-check my numbers by looking around at the NAEP and DC OSSE websites — which are unfortunately not easy to navigate, so good luck, and be persistent! You will also find that some years have two different scores reported, which is why I put those double asterisks at a couple of places on those curves.)

But here’s what’s really suspicious: the DC-CAS scores, shown in red, seem to jump around wildly and appear to show tremendous progress overall but also utterly un-heralded drops.

Which is it?

Slow, steady progress since 1996, or an amazing jump as soon as Wonder Woman Rhee comes on the scene?

In my opinion, I’d much rather trust the feds on this. We know that there has been all sorts of hanky-panky with the DC-CAS, as repeatedly documented in many places. I know for a fact that we math teachers have been changing the ways that we teach, to be more in line with the 1989 NCTM standards and the ways that math is tested on the NAEP. It’s also the case that there has been significant gentrification in DC, with the proportion of white kids with highly educated parents rising fairly steadily.

Slow improvement in math scores, going back a quarter of a century, makes sense.

Wild jumps don’t seem reasonable to me at all.

On the contrary, besides the known mass cheating episodes, it almost seems like DC education officials get together with McGraw-Hill CTB, which manufactures the DC-CAS, and decide how they want to get the scores to come out. THEN they decide which questions to count and which ones NOT to count, and what the cut-off scores will be for ‘advanced’, ‘proficient’ and so on.

Next time: 8th grade math; and 4th and 8th grade reading.

=======

Links to my other articles on this:

Part One  (fourth grade math)— this one right here

Part Two (8th grade math)

Part Three (all reading)

Just how flat ARE those 12th grade NAEP scores?

Perhaps you read or heard that the 12th grade NAEP reading and math scores, which just got reported, were “flat“.

Did you wonder what that meant?

The short answer is: those scores have essentially not changed since they began giving the tests! Not for the kids at the top of the testing heap, not for those at the bottom, not for blacks, not for whites, not for hispanics.

No change, nada, zip.

Not even after a full dozen years of Bush’s looney No Child Left Behind Act, nor its twisted Obama-style descendant, Race to the Trough. Top.

I took a look at the official reports and I’ve plotted them here you can see how little effect all those billions spent on testing;  firing veteran teachers; writing and publishing new tests and standards; and opening thousands of charter schools has had.

Here are the tables:

naep 12th grade reading by percentiles over time

This first graph shows that other than a slight widening of the gap between the kids at the top (at the 90th percentile) and those at the bottom (at the 10th percentile) back in the early 1990s, there has been essentially no change in the average scores over the past two full decades.

I think we can assume that the test makers, who are professional psychometricians and not political appointees, tried their very best to make the test of equal difficulty every year. So those flat lines mean that there has been no change, despite all the efforts of the education secretaries of Clinton, Bush 2, and Obama. And despite the wholesale replacement of an enormous fraction of the nation’s teachers, and the handing over of public education resources to charter school operators.

naep 12th grade reading by group over time

 

This next graph shows much the same thing, but the data is broken down into ethnic/racial groups. Again, these lines are about as flat (horizontal) as you will ever see in the social sciences,

However, I think it’s instructive to note that the gap between, say, Hispanic and Black students on the one hand, and White and Asian students on the other, is much smaller than the gap between the 10th and 90th percentiles we saw in the very first graph: about 30 points as opposed to almost 100 points.
naep 12th grade math by percentiles over time

 

The third graph shows the  NAEP math scores for 12th graders since 2005, since that was the first time that the test was given. The psychometricians atNAEP claim there has been a :statistically significant” change since 2005 in some of those scores, but I don’t really see it. Being “statistically significant’ and being REALLY significant are two different things.

*Note: the 12th grade Math NAEP was given for the first time in 2005, unlike the 12th grade reading test.

naep 12th grade math by group over time

 

And here we have the same data broken down by ethnic/racial groups. Since 2009 there has been essentially no change, and there was precious little before that, except for Asian students.

Diane Ravitch correctly dismissed all of this as a sign that everything that Rod Paige, Margaret Spellings and Arne Duncan have done, is a complete and utter failure. Her conclusion, which I agree with, is that NCLB and RTTT need to be thrown out.

 

Trends for DC & Charters & Nation in 8th grade NAEP reading scores, black students

Here we have yet another surprising graph showing how the scores for black 8th graders on the NAEP reading tests have been bouncing around for students in DC public schools, DC charter schools, DC as a whole, large US cities as a whole, and the nation’s public schools as a whole.

Tell me what you see:

dc, dcps, charters, national, black 8th grade reading, naep to 2013

What I see is that under the ‘leadership’ of Rhee and Henderson, African-american 8th graders enrolled in DC public schools (blue and purple line) are actually doing a bit worse than they did before mayoral control. However, the average scores for the their counterparts in DC’s charter schools (dotted orange line)  are rising quite rapidly and are now higher than the national averages for black 8th graders.

However, on the average, the scores for all 8th-grade black students in publicly-funded DC schools (black dashed line) on the NAEP since 2008 (when Rhee was installed – purple vertical line) seem to be following the trends that were in place before that date.

No wonder Henderson recently admitted that her administration had no real idea on how to make DCPS middle schools attractive to families. One might conclude that the DC African-American families and students who were motivated to do well in school have in many cases migrated to the charter schools, leaving the less-motivated ones behind.

As in my previous three posts, I had to do have my spreadsheet do some computation to calculate the scores for the charter schools. You can find the formula in my first two posts. I used the overall DCPS and charter school and DC total enrollments rather than the specific 8th-grade-level enrollments for each institution because the latter was too difficult to find and I suspected that it wouldn’t make a big difference. If anybody finds any errors, please let me know.

Trends on NAEP for 8th grade math, black students in DC, DCPS, DC charters, and nation

Yet another graph, this one showing how this year’s group of African-American 8th grade students did on the NAEP math tests in the regular DC public schools, in all DC publicly-funded schools, in the DC charter schools, in large cities across the nation, and in all US public school systems, going back to the early 1990s.

dc, dcps, charters, national - black 8th graders, math to 2013

As usual, I had to do a bit of algebra to calculate what the average charter school scores were in the post-Rhee era, since those are not explicitly given anywhere. I give the explanation in my previous two posts.

My previous results seem to disagree a bit with those produced by NCES (by a couple of points). Therefore I used their data instead of what I calculated; the graph above is new as of 1/6/2014.

I still make these conclusions:

(1) Since the establishment of mayoral control of the schools, as a whole, the overall average for DC students in publicly-supported schools is following just about the exact same trends that were established from 2000 through 2007.  As a result, math scores for DC’s African-American 8th graders are now equal to those in large cities across the nation, which is a positive development.

(2) The DC charter schools seemed to have siphoned off the more motivated black 8th grade students and their families; as a result, scores for students in the regular DC public schools at that level in math lag significantly behind those of their counterparts in the charter schools, whose scores now surpass those of black 8th graders n the nation’s public schools as a whole and also those in large urban school systems as well.

As usual, if anybody finds any errors in my work, please let me know by leaving a comment.

Trends in DC on the NAEP for 4th grade reading, black students only: regular DCPS, charter schools, and pre- and post-Rhee

Here is a graph showing how African-American 4th students have been doing over time in Washington DC public schools and charter schools. I have drawn a clear dividing line at year 2008, because the scores before that were under the influence of DC’s former school board and superintendents. After that time, DC has been under a chancellor answerable only to the mayor.

dc, dcps, dc charter, and national naep trends, 4th grade reading to 2013You may notice that the blue, black and purple lines separate after 2007. That’s because NAEP began reporting separate scores for DC’s regular public schools and for all publicly-supported schools, though not for the charter schools as a bloc. As a result, you have to do a little bit of linear algebra to calculate what the average scales were for the charter schools from 2009 onwards. (I used essentially the same equation that I did in the previous post. Please write me a note if you think I made an error.)

As usual, we can see that since the late 1990s and up until Rhee took over, the overall trend in all large cities, in the nation’s public schools, and in DC’s publicly-supported schools was upwards on this test. (Yes, I know, these are not scores that follow the same kids year after year, but for whatever reason, the group of kids answering these tests are in general getting more answers right every two years.) Before that, i.e. from 1992 to 1998, scores bounced around or went down.

After Rhee took over, those scores seem to have entered another bouncy period. In fact, in DCPS, the scores on this test in 2013 were only back up to the level of 2007. There is a clear demarcation between the scores in the charter schools (blue line) and the regular public schools. The line for the charter schools seems to follow the trend from 1998 to 2007.

If I knew nothing about the politics of EduDeform, I would wonder why the WaPo editorial board is claiming victory.

 

 

Trends in DC’s regular public schools and charter schools: 4th grade math NAEP, TUDA

I continue here in showing you the results of my calculations for how the charter school students and regular public school students in Washington, DC have been faring on the National Assessment of Educational Progress, or NAEP, since the 1990s.

Some of my previous columns were quite simple: I just cut and pasted graphs from the NAEP and NAEP TUDA results, or asked the built-in software for how white, black, hispanic, special education, or free/reduced-price-lunch kids did at the 4th and 8th grade in math and reading.

If you look at my previous graphs, you will notice that, on the whole, the trends AFTER 2007, when Michelle Rhee was installed as the very first DC chancellor, looked just about the same as the trends BEFORE that date.

Today, I did a little math to figure out how black fourth-grade charter school students did in math in DC, in comparison with their counterparts in other large cities, in the nation as a whole, and in the regular DC public schools.

The math goes like this: I figure that the DC state weighted average for any given group or grade level (say, 4th grade African-American students taking the math NAEP) equals the weighted average for regular DCPS at that grade level, times the enrollment at that grade level, plus the product of the charter school weighted average score at that grade level and the charter school enrollment at that grade level; all of that divided by the total enrollment.

Or, if Q = DC state average. and R = DC regular public school weighted average, and V = DC regular public school enrollment, and S = DC charter school weighted average, and W = DC charter school enrollment, and X = V + W = total enrollment in publicly-funded schools in DC, both regular and charter, then

Q = (R*V + S * W) / X

And since I could find everything except S in the literature, then I could simply solve for S. My result:

S = (X*Q – R*V)/W.

And here are my results:

dc, dcps, charters, national - black 4th graders, math, naep, 1996-2013

 

My conclusions?

For black students at the 4th grade in math, the post-Rhee trends in the charter schools are about the same as the trends in DC public schools were BEFORE Rhee was appointed. However, it looks like the trends overall in the regular public schools seem a bit worse.

If past trends had continued, and Michelle Rhee had not become chancellor, the overall educational results might have been very similar to what they are today — inequalities and inequities of course included, because we have lots of that here in Washington, DC.

By the way, if anyone finds a mistake in my work, please let me know by leaving a comment.

Has Educational Rhee-form succeeded or failed in Washington DC Public Schools?

Bottom line conclusion from my last bunch of posts (see #1, #2, #3, #4, #5, #6)

Mayoral control of schools in DC, aka Educational Deform à la Rhee, has been an expensive failure, and it was foisted on us under false pretenses.

How can I make that conclusion?

Very simple.

This Rhee-form has fulfilled none of its promises, even on its own terms.

Its backers (Gray, Rhee, Henderson, Duncan, Bloomberg et al) claim that it’s been a great success.

But if you look at the graphs, it is clear that if the regime of Rhee and Henderson is going in the right direction, then so was the previous DCPS regime under superintendents Janey and his predecessors.

Any good trends have continued mostly unchanged.

Remember that we were promised incredible gains in test scores? Compared with the ‘bad old days’ when teachers actually had the right to due process before being fired? And back when poor DC students still had recess and PE and art and music libraries? And compared to the evil era when their teachers weren’t required to waste nearly the entire year on scripted test-prep lessons?

None of those incredible gains show up in the data, any more than they did when Michelle Rhee wrote all those lies in her resume. (I mean, why does ANYBODY listen to a liar like that, or to Rob Ford, or to Michael Millken or Bernie Madoff or the CIA/EPA liar?)

Anybody claiming that the last six sets of NAEP  TUDA scores show brilliant success for educational Rhee-form is engaging in wishful thinking or lobbying.

What’s more, my previous posts (and those of several other researchers and commentators) have shown that there is essentially no correlation between Value-Added scores and anything else. So that’s a failure, even on its own terms: it predicts nothing, it doesn’t help teachers teach better, and is essentially a random-number generator that clearly has done nothing to improve educational outcomes in DCPS, even though it costs taxpayers many, many millions of dollars and consumes a tremendous amount of time – something teachers and other staff have far too little of.

Mayoral control  has lived up to exactly NONE of its promises of closing the achievement gap between the ‘haves’ and ‘have-nots’, or of improved testing outcomes for the students of the District of Columbia any better than did the pre-Rhee superintendent-and-school-board system.

Trends are almost exactly the same now as they were before Mayor Fenty got control of the schools and appointed that serial self-promoter, liar and distorter of facts, Michelle Rhee, as chancellor of DC public schools, where she led an assault on the system which has fired or forced out many thousands of teachers, producing a revolving door of constantly churning teachers who are in turn forced out or fired. What’s more, Rhee-form has turned over half the public school system to private operators with no accountability (some of them brazen criminals) and track record of success except by exclusion and undemocratic practices. Rhee-form has also subjected all students in DCPS to a stultifying test-prep regime where arts, music, social studies and recess are banned and principals themselves can be canned at any time and are under incredible pressure to cheat and get rid of teachers.

From everything I have seen, it is not at all difficult to be doing your job as a teacher just fine, and end up with a mysterious numerical score known as IVA based on some unexplained formula that gets them fired. People have confessed to me that they were wholly unable to teach at all because kids were figuratively running wild in their classrooms, yet they got great “Value-Added” scores anyway. Teachers who became National Board Certified, a tremendous accomplishment, told me of some years (but not others) getting IVA scores so low that it would put their job at risk.

Anybody claiming that the data trends before 2008 look different from the ones after 2008 is engaging in wishful thinking.

So, if Kaya Henderson and Vincent Gray and Arne Duncan claim that the current policies are causing recent gains, then they logically must conclude that the previous policies were producing the same results, and should have been continued as well.

It’s a big, expensive lie that has had real consequences.

Students are wasting nearly an entire school year under stultifying, scripted lessons preparing for an ever-lengthening regime of utterly stupid and poorly-prepared but highly secret standardized tests whose manufacturers are responsible to no-one except their billionaire CEOs. In fact, for the high-stakes tests, it’s considered cheating for the teachers even to analyze the tests after they are given, and results aren’t available until the end of summer, even though it’s a machine-scoreable test which in theory could have a good part of it be graded and fully tabulated in mere seconds… that is if the publishers actually knew what they were doing and weren’t busy lobbying among themselves as to what mathematical and sleight of hand tricks they would play with the data to make it come out the way that the politicians they want…

Follow

Get every new post delivered to your Inbox.

Join 463 other followers

%d bloggers like this: