More on the DC Education Frauds

This article appeared in Education Week, which is behind a paywall, so I’m pasting it here. In case you haven’t been watching, just about all of the supposed improvements in DC’s publicly-funded education sector have either been:

(a) continuations of trends begun before Mayor Fenty took control of DC Public Schools in 2007 and appointed Michelle Rhee Chancellor; or

(b) the result of changing demographics (more white kids, more black kids from relatively-affluent families, and fewer kids from highly-poverty-stricken families; or

(c) simply the result of fraud.

========================================

NEWS

D.C.’s Scandal and the Nationwide Problem of Fudging Graduation Numbers

Edweek.org

The headlines made a big splash, and yet they were strangely familiar: Another school system was reporting a higher graduation rate than it deserved.

The most recent scandal-in the District of Columbia-is just the latest example in a growing case file of school systems where investigators have uncovered bogus graduation-rate practices.

Those revelations have unleashed a wave of questions about the pressures and incentives built into U.S. high schools, and fueled nagging doubts that states’ rising high school graduation rates-and the country’s current all-time-high rate of 84 percent-aren’t what they seem.

The newest round of reflections was triggered by an investigation, ordered by the D.C. mayor’s office, that found that 34 percent of last year’s senior class got diplomas even though they’d missed too much school to earn passing grades, or acquired too many credits through quick, online courses known as credit recovery. Only three months earlier, the school system touted a 20-point rise in its graduation rate over the last six years.

“It’s been devastating,” said Cathy Reilly, the executive director of the Senior High Alliance of Parents, Principals, and Educators, a group that focuses on high school issues in the District of Columbia. “It’s made people here feel that our graduation rate gains weren’t real.”

A National Problem

Such revelations are hardly confined to the nation’s capital. In the last few years, a federal audit found that California and Alabama inflated their graduation rates by counting students they shouldn’t have counted. News media investigations showed that educators persuaded low-performing students in Atlanta and Orlando, Fla., to transfer to private or alternative schools to eliminate a drag on their home schools’ graduation rates.

See AlsoThe D.C. Public School Attendance Scandal: Where’s the Outrage? (Commentary)The drumbeat of graduation-rate fudging has opened the door to renewed attacks on the pressures imposed on schools by accountability rules, particularly the high stakes that some systems attach to specific metrics. In the District of Columbia, for instance, high school teachers and principals are evaluated in part on their schools’ graduation rates.

With those kinds of stakes, teachers can feel immense pressure to award passing grades to students who haven’t earned them, a dilemma that intensifies in schools with high rates of chronic absenteeism and academically struggling students.

In a survey of 616 District of Columbia teachers conducted after the scandal broke, 47 percent said they’d felt pressured or coerced into giving grades that didn’t accurately reflect what students had learned. Among high school teachers, that number rose to 60 percent. More than 2 in 10 said that their student grades or attendance data had been changed by someone else after teachers submitted them.

Scott Goldstein oversaw the survey as the founder of EmpowerEd, a year-old coalition of D.C. teachers that works to strengthen teacher leadership. To him, the results cry out for a new conversation about the “moral dilemmas” embedded in accountability systems that rely heavily on just a few metrics, like graduation rates.

“If you pass students [who haven’t completed course requirements], you’re leading them into a world they’re unprepared for. But if you fail them, you’re harming their lives in other ways,” said Goldstein, a social studies teacher at Roosevelt High School. Teachers’ decisions should rest on a professional appraisal of student mastery, not on fear for their own jobs, he said.

Pressure From the Top

Pressure to Graduate: Perspectives From Educators … read moreEven in school systems that don’t reward or penalize educators for their schools’ accountability metrics, teachers can feel immense pressure from administrators on their grading practices.

In postings on social media, Education Week asked high school teachers if they’d ever felt pressure to give passing grades to students who hadn’t done the work.

“Never mind high school. I feel that pressure in 3rd grade,” said Annie, an elementary school teacher in central Virginia. She asked Education Week not to identify her so she could discuss sensitive issues.

She said her principal has cautioned her not to fail any student or recommend that they repeat a grade because she “doesn’t want anyone to feel bad about not succeeding.” When she gave a student a D recently, she was summoned to a meeting with the principal, Annie said.

“She was upset. She said, ‘Why didn’t you work harder to get the student to turn in missing work, or re-do work?’ She sees a D as a teacher’s failure. But I think it’s a disservice to kids to give them grades they haven’t earned.”

John R. Tibbetts, who teaches economics at Worth County High School in rural Sylvester, Ga., and is the state’s 2018 teacher of the year, said his district’s policy doesn’t include course-failure rates in teachers’ evaluations. But his principal recently sent teachers an email conveying word from their superintendent that “failure rates … will be taken into consideration” in their evaluations anyway.

A Change of Approach

Tibbetts said he would like to replace that “threatening” posture with a more collaborative one.

“If the superintendent is concerned with course-failure or graduation rates, what we really need is for him to have a conversation with teachers about what we need to do to improve, what policies we can implement,” he said.

Education advocates who believe accountability can be a force for good worry that graduation-rate scandals could tarnish a tool that’s important for shining a light on inequities and applying pressure for school improvement.

They hope, instead, that uncovering problems can spark a rebalancing of the pressures and supports built into accountability systems, and change school practice to respond better to issues like students’ poor academic skills and chronic absenteeism.

“We shouldn’t stop paying attention to high school grad rates, or not have them in accountability systems,” said Michael Cohen, the president of Achieve, which works with states to raise academic expectations.

“The right response to all of this is to double down on efforts to support students, and to support teachers, early and consistently, so they’re not pressured to game the system and they can give kids what they need.”

Experts who study and track graduation rates acknowledge that in some places, the rates are inflated by cheating or inaccurate reporting. But they contend that those cases account for a tiny share of schools overall. Robert Balfanz, a Johns Hopkins University researcher who studies graduation rates, estimates that those cases account for 2 to 4 percentage points in the national graduation rate.

‘Hard-Earned Gains’ Are Real

John Bridgeland, the chief executive officer of Civic Enterprises, a think tank that examines graduation rates for the annual “Grad Nation” reports, said his team has visited dozens of schools to find out what they’re doing to produce significant gains in their graduation rates.

In a few places, he said, he and his colleagues have had to shave 2 to 4 percentage points off the rates districts were reporting because they were improperly counting some types of students who shouldn’t be included, such as those who started home schooling in their junior year of high school.

But with few exceptions, Bridgeland said, his team has found that “the hard work” of better instruction and student support explains higher graduation rates.

“We need to call out the problems when gaming or cheating appears,” he said. “But at the same time, taking isolated examples of gaming the system and saying that high school grad rates are not real diminishes and undermines the many schools, districts, and states that have hard-earned gains and clear progress to showcase,” he said.

Those who study graduation-rate calculations point out that while they’re still imperfect, they’ve been much more reliable since 2008 when federal regulations began requiring all schools to calculate them the same way-the portion of each freshman class that earns regular diplomas four years later.

Balfanz said that more stringent calculation and reporting requirements “without a doubt” have been responsible for a very real rise in states’ graduation rates.

“People don’t remember the bad days before 2008, when schools were allowed to measure graduation rates however they wanted,” he said. “Kids dropped out, schools would code them as ‘whereabouts unknown,’ not as a dropout. No one knew, and no one cared. That wasn’t a good place. Accountability makes schools pay attention to a key outcome, like graduating our kids from high school.”

But even those experts acknowledge that there are still too many hidden variations in the way states report graduation-rate data. To get a more accurate understanding of schools’ graduation rates, they’ve quietly identified about a dozen variations that should be ferreted out and handled in uniform ways.

For example, even though federal rules don’t allow states to count summer graduates, or those who earn high school equivalency certificates, some still do. Some schools include summer graduates, or students in juvenile justice facilities. Others include teenagers who “transfer” into home schooling late in high school.

What’s Behind the Record Rises in U.S. Graduation Rates?

Education Week
New Federal Rule Could Force States to Lower Graduation Rates

Education Week
NCLB Rules Back Common Rate

Advertisements

Where DC’s schools rank by family income, test scores, and ethnicity – NYTimes

The New York Times recently ran the results of some pretty fancy number-crunching for all sufficiently-large public school districts in the United States. They graphed family income against ‘years ahead or behind’ in school and also showed the discrepancies in each of those school districts among hispanics, whites, and blacks.

If you haven’t played with the graphs, I urge you to do so. I did a little bit, looking for Washington, DC, my home town, where I and my children attended and where I taught for 30 years. I already knew that DC has one of the largest black-white gaps anywhere in the nation – a gap that 9 years of Edu-Reform under Fenty, Rhee, Gray, Henderson various charter companies have not narrowed at all.

Notice the extremely tight correlation between family income and scores on achievement tests, and where the District of Columbia is situated on the graph.

disparities dcps nyt

This next plot shows where DC’s whites, hispanics, and blacks are situated on the graph (as well as for thousands of other school districts):

Disparities dcps wh blk his nyt

Notice that white students in DC’s public schools are nearly the wealthiest and highest-achieving group anywhere in the nation, while DC’s black students are very far behind in both income and achievement. DC’s hispanic students, to my surprise, are considered to be a bit above the middle of the income levels, but still rather far behind academically. (I actually rather doubt the data on those DC hispanic income levels, based on my own personal experiences with Hispanic families here in DC…)

Even the Chancellor Calls the Results ‘Sobering’

The Washington City Paper has an article on the PARCC results with way more graphs and charts than I do, and they quote even Chancellor Kaya Henderson as saying the results were ‘sobering’.

Please remind me why she still has a job?

She and several other speakers said that the PARCC results were more ‘honest’ than the old DC-CAS results, probably because the new ‘passing’ scores are lower than the old ones. I guess that means that it’s more ‘honest’ to say that students are doing worse than we were previously led to believe, under the current regime of all-testing-all-the-time and turn-half-the-students-over-to-unregulated charters?

 

PARCC Results Released in DC

I just got back from watching the public release of the results of the PARCC test that students in Washington DC took about 7 months ago.

(Let that sink in: it took the testing company, and their consultants, and the city’s consultants, over HALF A YEAR to massage the data into a releasable form. So much for having these tests be able to be used to ‘inform instruction’ or help teachers figure out what kind of help their students need. It’s now the last day of November, and the students have been in school since August. What kind of help is that to teachers or parents? And tho I haven’t looked at the released school scores or samples of what the teachers will see, I’m not optimistic. If the past is any guide, the scores themselves will be essentially useless as well…)

(It won’t take so long next time, we were assured…)

I got to see Mayor Bowser, Councilman Grosso, Chancellor Kaya Henderson, [powerless] Superintendent Hanseul Kang, and Deputy Mayor for Education Jenny Niles, and charter honcho Scott Pearson perform and answer some mostly-lame questions from some members of the media.

What we saw were that advanced students in DC (largely white ones) do exceedingly well on this PARCC battery of tests, and that others (blacks; hispanics; SPEDs; students on free or reduced lunch; ELLs; or Students At Risk) do much worse. Which of course is  the very same result we’ve seen on the NAEP for a couple of decades.

In fact, of all the cities and states measured on the NAEP, Washington DC has the very widest gaps in test scores between the Upper Caucasia Haves and the Have-Nots everywhere else, and those gaps are if anything getting wider.

It was interesting to hear Henderson’s defenses of the results, which still showed very low percentages of most students “passing” the PARCC. She said, among other things, that

(1) since students at the lower grades generally scored higher than those at the upper grades, that show’s we are on the right path [seems to me it shows the exact opposite; the longer that students have been exposed to “Reform”, the worse they do… and

(2) It takes a long time, you can’t just expect to turn a switch and have everything be wonderful overnight, we need lots of wrap-around services and a longer school day and school year and support for teachers.

Regarding the latter excuse: isn’t that exactly what teachers were condemned for saying under Chancellor Rhee, whose understudy was none other than Kaya Henderson? Didn’t Rhee imply that the only reason that poor students did poorly in school was that their greedy, lazy teachers, empowered by their evil union, refused to teach them anything? And that anybody who said that it’s a lot harder to teach impoverished students of color with chaotic families (if any) than it is to teach middle-class children with educated parents – why those people were just making excuses for poverty?

 

Six more promises from Rhee, Henderson & Company: Were ANY of them Kept?

Part Five of Many

You may recall that Michelle Rhee is famous for summarily firing administrators who didn’t bring student scores up the way she demanded.

As a result, the turnover in principals in DCPS is phenomenal.

The 2010 contract between DCPS and a bunch of billionaire-funded foundations also called for penalties if certain targets were not met.

I’m systematically going through those targets to see how many (if any) of those targets were actually met.

So far, out of four targets, Rhee/Henderson et al have ALMOST met one of them.

We have six more targets today, concerning specific numerical targets to be reached on the National Assessment of Educational Progress (NAEP) and its Trial Urban District Assessment (TUDA) from about 2009 through 2013.

For a change, I’ll start with 8th grade reading, then fourth grade reading. Next time, the same grade levels in math. Try to keep score as we go along.

8th grade naep tuda targets 2007-13

 

Let me try to explain what’s going on here. In 2007, the average score for all DCPS 8th graders in reading on the NAEP TUDA was 241. Rhee and her accomplices in DCPS promised that in 2009, that score would rise to 244, then to 248 in the year 2011, and finally hit 252 in 2013 (last year); those are the blue dots and lines that you see in the graph and in the chart.

In fact, the actual average NAEP TUDA scores were significantly lower, in each and every single year. In 2009 the average was actually 240, then 237 in 2011, and 245 last year (2013).

In other words, not a single one of those targets were met.  I count that as zero for three.

You may notice that I drew brown ‘diamonds’ (rhombi) around the cases where the targets were not met.

Now let’s look at 4th grade reading:

4th grade naep tuda targets 2007-13In this case, one of the targets was actually met! Huzzah! In the year 2009, the prediction was that the average scale score for DC public school students would be 202, but it was actually 2003! Yowzers!

But in 2011 and 2013, the goals were NOT met. So that’s only 1 for three.

=================

I think I’ll put the math scores in an additional post, since this is getting pretty long.

But let’s keep score overall. Today, Rhee’s record is 1 for 6.

Over the last few days, we found that she got ALMOST a 1 out of 4.

Overall, that’s about one and a half out of ten, or 15 percent.

Anywhere in the US school system, and everywhere else that I know of, that is a very solid failing grade.

So, why does Kaya Henderson* still have a job as DCPS chancellor, and why does anybody listen to any of the promises that Michelle Rhee makes and has made, when their record is almost unremitting failure — EVEN ON THEIR OWN TERMS?

=======

Thanks to Erich Martel for bringing this data to my attention and compiling the actual average scale scores from the NAEP TUDA website!

* Right-hand henchperson of Michelle Rhee and her acolyte and booster at “The New Teacher Project” and a failed NYC teach-for-a-while many years ago.

==============

Alert readers will have noticed that I originally added wrong (three plus three is generally six), so I gave Rhee and company a better score than they deserved. This semi-alert writer caught his own mistake before anybody else rubbed his nose in it. Her score is 1.5 out of 10 (15%) overall, not 1.5 out of 9 (about 17%). So far. There are many more goals to go!

================

The saga so far:

  1.  https://gfbrandenburg.wordpress.com/2014/09/02/did-any-of-michelle-rhees-promises-actually-work-in-dc/
  2. https://gfbrandenburg.wordpress.com/2014/09/02/more-on-michelle-rhees-promises-concerning-dcps/
  3. https://gfbrandenburg.wordpress.com/2014/09/04/what-rhee-promised-to-the-billionaires-walton-gates-et-al-but-didnt-deliver/
  4. https://gfbrandenburg.wordpress.com/2014/09/04/two-more-promises-by-rhee-et-al-were-they-kept/
  5. https://gfbrandenburg.wordpress.com/2014/09/05/ten-more-promises-from-rhee-henderson-company-were-any-of-them-kept/   (this one)
  6. https://gfbrandenburg.wordpress.com/2014/09/05/33-6-for-nearly-all-values-of-3-not-5/
  7. https://gfbrandenburg.wordpress.com/2014/09/05/5281/
  8. https://gfbrandenburg.wordpress.com/2014/09/07/more-failures-to-deliver-on-promises-by-michelle-rhee-and-her-acolytes/
  9. https://gfbrandenburg.wordpress.com/2014/09/08/another-day-another-bunch-of-failures-from-rhee-henderson/
  10. https://gfbrandenburg.wordpress.com/2014/09/09/even-more-missed-targets-dc-cas-proficiency-in-2010-and-2011/
  11. https://gfbrandenburg.wordpress.com/2014/09/13/rhees-failures-in-dc-the-continuing-saga-2012-dc-cas/
  12. https://gfbrandenburg.wordpress.com/2014/09/21/the-long-list-of-failures-by-rhee-and-henderson-continued/

\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

Once again, let me credit my colleague Erich Martel for coming up with the idea of going back to the original promises and seeing if they were kept or not, and sharing his findings with me. These calculations are generally my own, so if you find any mistakes, don’t blame him. Blame me.

Latest DC Audited Enrollment Figures for all, charters, and regular public schools

The latest audited enrollment numbers have just been released, but not in a very useful format.

They show that regular DCPS enrollment is pretty close to flat, with only a small change over last year, or even over the last seven years. However, overall enrollment in all taxpayer-funded schools in the District of Columbia continues to rise, mostly because of a steady 15-year-long rise in charter school enrollment and a large increase in the overall city population.

The strangest feature I see is that the high school enrollment (grades 9-12) is down at all types of schools, with apparently many of those students moving to ‘alternative’ schools, at least on paper.

As I said, I didn’t think the graphs put out by OSSE were very informative, so I’ve re-plotted them here. For example, they put the charter school and public school enrollments on different graphs with different scales, making them hard to compare.

My first graph is of overall enrollment figures for regular public schools and for the charter schools (which several courts have decided are NOT public entities)  since the start of the millennium:

audited enrollment, dc public and charter schools, 2001-14

 

The red line is enrollment in the charter schools, and the blue line is that of the regular public schools. You can see that the blue line has been just about level since 2007-8, when Michelle Rhee was appointed chancellor of DCP.

My next graphs explores where the students are. OSSE separates students into various “bands” which are a bit hard to decipher. PreK3, PreK4, and Kindergarten totals are counted separately, and then they lump together grades 1-3 (‘primary’), then grades 4-5 (‘upper elementary’), then grades 6-8 (‘middle’), and grades 9-12 (high school). Students in alternative schools, of unspecified ages, are counted separately, as are students enrolled in Special Education schools and those in adult learning centers.

This first one is for regular DC Public Schools. You can see that preK3 though grade 3 comprises just under half of the entire DCPS population.

overall dcps - only enrollment by bands, 2013-4

 

The next graph shows the same thing but for ALL taxpayer-funded schools, both public and charter. Notice that the ‘adult’ sector is larger here.

overall dc osse enrollment by grade bands, 2013-4And the next graph shows the same thing for just the charter schools:

overall dc charter enrollment in percentages by grade bands 2013-4We see a much larger fraction of students in the adult sector. Again, Prek3 through grade 3 makes up just under half of the total.

Now let’s look a bit closer at the changes from last year to this, by grade band. My first graph shows overall changes from last year to this year, in all taxpayer-funded schools in Washington DC. Notice the large increase in the ‘alternative’ population and the ‘adult’ population, followed by a somewhat smaller rise in grades 1-3. The high school population – both public AND charter – actually dropped, as did the number of students enrolled in a special education school like Sharpe. It appears that a large fraction of that drop is students being reclassified as “alternative” instead of being in a high school.

increases, decreases by grade level, all DC OSSE schools, 2012-3 ri 2013-4Now let’s look at the corresponding graph for the regular DC public schools:

actual increases or decreases by grade level, DCPS only 2012-3 to 2013-4

 

Notice that once again, there was a big jump in the ‘alternative’ population, followed by an increase of about 250 at grades 1-3. As in overall DC stats, there was a drop in grades 9-12 and in special education. (the number for grade 6-8 is a typo: it should be 50)

Lastly, here are the changes since last year by grade band for the DC charter schools:

actual changes in enrollment, dc charter schools by grade bands, 2012-3 to 2013-4

 

I was surprised to see small drops in all of secondary charter schools (that is, grades 6 through 12). We see robust increases at all other levels, especially at the adult and alternative levels. I’m not exactly sure what’s causing this; perhaps readers closer to the trench lines than me (retired 5 years now) can comment.

My understanding  from reading US census figures is that the number of teenagers in Washington, DC – and thus, the number of students eligible to enroll in grades 6-12 continues to fall, while the number of younger kids is increasing. Obviously, those little kids generally grow older, and soon we will see a robust increase in the high school enrollment in the public and charter schools — unless they and their families all move out of town or decide to spend huge amounts for private or parochial schools. Which I doubt will happen.

In any case, claims of huge increases in enrollment in the DC public schools under chancellors Henderson and Rhee are just wishful thinking — like most of the boasts on Michelle Rhee’s famous resume.

 

Comments are most definitely welcome, even if you need a magnifying glass to see the “comments” button.

 

 

 

 

Published in: on February 28, 2014 at 3:41 pm  Comments (2)  
Tags: , , , , , ,

Has Mayoral Control of Schools in DC Succeeded or Failed? A 10-year Record of Data

In the past few posts, (#1, #2, #3) I’ve merely cut-and-pasted graphs or text that the National Center for Educational Statistics (NCESpublished two days ago on their website as part of what has become a very widespread, longitudinal survey of statistics illustrating educational progress or problems in the large urban school systems of the USA; this one was called the NAEP TUDA.

The only modifications I did to the graphs were cutting out the ‘fine print’ that few of my readers would look at anyway, and adding some notations and color for clarity.

(BTW, if you want to look at the fine print, go right ahead! It’s all findable at that same website.)

This time, I actually lifted a few digital fingers and asked the NCES web site to produce some simple tables of longitudinal and cross-sectional data for DCPS and a few other cities. Nothing complicated: I just wanted to see how black, white, and Hispanic  students have been faring, along with those in special education and immigrant kids learning English for the first time, compared with ‘regular’ kids, and compared with tbejr counterparts among alloother large public urban school systems. Then I had Excel plot the data. Here are my first two such graphs, exploring whether mayoral control of schools (and all that went along with that, such as eliminating tenure and turning education over to private corporations) helped or hurt.

I think the results are obvious.

Here goes:

4TH GRADE math naep tuda scale scores for black, white, hisp - nation + DCPS

 

Lots of information in this first graph, but you have to pay a little attention.

(1)    Notice how high white students in DCPS score on this graph (solid red line, at the top). Those students, some of whose siblings I’ve taught at Alice Deal JHS/MS, are the highest-scoring subgroup that I know of in the entire NAEP/NCES/TUDA database. Overall, they continue to do well, and in the fourth grade, for math, the only departure from a straight-line trend was 2007. Their rate of growth in test scores exceeds that of white kids in all urban public school systems, probably because white kids in DC overwhelmingly come from professional, educated families. We don’t have any trailer parks or other sizable population of white working-class kids in DC, ever since the massive “white flight” of the 1960s. (You have to go elsewhere to find characters like those on “Honey Boo Boo”!)  In any case, overall, no real change pre-Rhee to post-Rhee, other than the fact (not apparent in this graph) that the proportion of white students at most grades has vastly increased in DCPS: in a word, because of gentrification. In any case, white kids in DC continue to score a lot higher in 4th grade NAEP math than white kids in other public urban school systems (dotted green line near the top).

(2)    Among Hispanic students, it appears that the trends after 2008 in DCPS for fourth-grade math students aren’t so favorable to the pro-mayoral-control side of the argument: from 2003 through 2009, their scores were increasing at a pretty amazing rate (solid purple line) until they matched the scores of Hispanic students in all US urban school systems (dotted purple line). After that year, those scores went down or leveled off. Again, no miracle.

(3)    Among black students at this grade level, if the trends for 2003-2007 had continued, the bottom orange line for black 4th-graders in math would be a bit higher than it is now, largely because there was in fact no growth from 2009 to 2011.

(4) It’s a bit harder to see, but the hispanic-white and black-white achievement gaps at this grade level continues to be a lot larger in DC than it is in the nation’s urban school systems. Twice as wide, in fact. So, again, no sign of success.

Overall: no evidence here whatsoever of any of the promised miracles. In fact, if anything, growth was a little worse, overall, after Rhee, than it was before Rhee.

Now let’s look at achievement levels for 4th grade math students with disabilities (ie special education), ELLs (English Language Learners) and those in regular education, both here in DC and in all US urban public school systems. Here, I chose to plot the percentages of students who are “Basic” or above, rather than the average scale score. You could plot scale scores yourself, if you like.

pct students basic or above DCPS and nation by disability - 4th grade math NAEP TUDA

(A score of “Basic” on the NAEP corresponds to “Proficient” on the DC-CAS and other state-administered NCLB and RttT tests.)

Notice that most of these lines show an overall upward trend for this period. The top line (dotted, green) is the percent of all public-school, regular-education students in urban public school systems who score “Basic” or above on the fourth-grade math NAEP. The  solid, maroon/brown line represents the same measurement for regular 4th-grade math students here in DC. Notice that both the dotted green and solid brown lines are going up pretty steadily, with no particular change in trend on either side of the vertical orange line. Which means that mayoral control seems not to have changed  to past trends one way or the other.

The olive-colored, dotted line represents percentages of fourth-grade students of English as a second language in all of our urban public schools. As you can see, the trend is a slow but steady increase. However, in DC public schools, since 2009, the corresponding line (solid, sky blue) is trending downwards. Why? I have no idea, but it’s not a favorable argument for continued mayoral control, since before Fenty and Rhee took over DCPS, the trend was certainly upwards.

With special education students, I used a dotted purple line for all national urban public school students, and a solid orange lines for those in DC public schools. I have no idea why the percentage of 4th-grade math students scoring “Basic” or above went down across the nation’s cities after 2008, while it had been going up modestly but steadily before that date. Clearly, the trends in special-education scores in DCPS are even more mysterious: a continuation of past trends in 2009, a fairly large drop in 2011, and a fairly large increase in 2013. In any case, if you were to extrapolate the orange pre-Rhee line past the central line, I suspect you’d come to about  the same place we are in right now (2013).

Again, this is evidence that all the churn, upheaval, anguish, money, and curriculum impoverishment of the past 6 years in the District of Columbia has all been for naught. We would have gotten the same results with the system we had in place beforehand.

 

My conclusion: any progress in DCPS appears to be a continuation of trends that show up very clearly as going back ten years, well before the DC city council, with the blessing of Congress, abolished the school board and handed control of the public schools over to a chancellor appointed and responsible to a mayor.

Published in: on December 20, 2013 at 2:57 pm  Comments (2)  
Tags: , , , , , , , ,

More on those supposedly wonderful DCPS NAEP TUDA scores…

In this post, let us look at how the District of Columbia Public Schools fared on the Trial Urban District Assessment sub-set of the National Assessment of Educational Progress. I think you will agree that there has been no significant change in trends if you compare the pre-Rhee era and the post-Rhee era, which we are in now. None of these graphs were made by me: I merely copied and pasted them from the NAEP website, and added a little color and a few labels.

The next graph shows the average scores on the NAEP for 8th-grade math for DC Public Schools and for all large-city public school systems in the US. You will have to look very hard to notice any change in slope for the lower, blue line, which represents DCPS, on either side of the orange vertical line, which separates the pre-Rhee era from the post-Rhee era.

dcps and large urban public schools math 8th grade

The next graph shows the average scores on the NAEP for 8th-grade reading in DCPS and all other large urban school systems. There has been no large change in either the national scores or the local DCPS scores since 2002, but I guess the best we can say that after two periods of small declines after mayoral control was imposed in 2007, the scores actually went up a bit in 2013 in DC. However, DCPS students on the whole are a little farther behind other urban kids now, under Chancellor Henderson, than they were at any time in the era before Rhee. But the changes are not very large or significant.

dcps and large urban public schools 8th grade reading

dcps and large urban public schools math 4th grade

The previous graph shows average Math NAEP scores for fourth-graders in DCPS and all other urban districts. Do you really see any big changes in the trends for DCPS scores? They have been going up rather steadily since 2003… It’s nice to see that DCPS kids seem to be catching up with those in other cities, but that was happening anyway.

My last graph in this post is for fourth-grade reading. It looks like I forgot to draw the vertical line separating the pre-Rhee and post-Rhee eras. Draw it in yourself. Do you see evidence of the supposed miracles that getting rid of 90% of the veteran teachers and school administrators, and hiring enormous numbers of inexperienced, highly-paid central-office administrators, has caused?

I surely don’t.

dcps and large urban public schools reading 4th grade

In a future post, I will actually dive a little deeper and ask how much of these changes (or lack thereof) are due to changing demographics….

I will also attempt to tease out how the privately-run charter schools in DC compare..

Published in: on December 19, 2013 at 1:37 pm  Comments (6)  
Tags: , , , , , ,

What I actually had time to say …

Since I had to abbreviate my remarks, here is what I actually said:

I am Guy Brandenburg, retired DCPS mathematics teacher.

To depart from my text, I want to start by proposing a solution: look hard at the collaborative assessment model being used a few miles away in Montgomery County [MD] and follow the advice of Edwards Deming.

Even though I personally retired before [the establishment of the] IMPACT [teacher evaluation system], I want to use statistics and graphs to show that the Value-Added measurements that are used to evaluate teachers are unreliable, invalid, and do not help teachers improve instruction. To the contrary: IVA measurements are driving a number of excellent, veteran teachers to resign or be fired from DCPS to go elsewhere.

Celebrated mathematician John Ewing says that VAM is “mathematical intimidation” and a “modern, mathematical version of the Emperor’s New Clothes.”

I agree.

One of my colleagues was able to pry the value-added formula [used in DC] from [DC data honcho] Jason Kamras after SIX MONTHS of back-and-forth emails. [Here it is:]

value added formula for dcps - in mathtype format

One problem with that formula is that nobody outside a small group of highly-paid consultants has any idea what are the values of any of those variables.

In not a single case has the [DCPS] Office of Data and Accountability sat down with a teacher and explained, in detail, exactly how a teacher’s score is calculated, student by student and class by class.

Nor has that office shared that data with the Washington Teachers’ Union.

I would ask you, Mr. Catania, to ask the Office of Data and Accountability to share with the WTU all IMPACT scores for every single teacher, including all the sub-scores, for every single class a teacher has.

Now let’s look at some statistics.

My first graph is completely random data points that I had Excel make up for me [and plot as x-y pairs].

pic 3 - completely random points

Notice that even though these are completely random, Excel still found a small correlation: r-squared was about 0.08 and r was about 29%.

Now let’s look at a very strong case of negative correlation in the real world: poverty rates and student achievement in Nebraska:

pic  4 - nebraska poverty vs achievement

The next graph is for the same sort of thing in Wisconsin:

pic 5 - wisconsin poverty vs achievement

Again, quite a strong correlation, just as we see here in Washington, DC:

pic 6 - poverty vs proficiency in DC

Now, how about those Value-Added scores? Do they correlate with classroom observations?

Mostly, we don’t know, because the data is kept secret. However, someone leaked to me the IVA and classroom observation scores for [DCPS in] SY 2009-10, and I plotted them [as you can see below].

pic 7 - VAM versus TLF in DC IMPACT 2009-10

I would say this looks pretty much no correlation at all. It certainly gives teachers no assistance on what to improve in order to help their students learn better.

And how stable are Value-Added measurements [in DCPS] over time? Unfortunately, since DCPS keeps all the data hidden, we don’t know how stable these scores are here. However, the New York Times leaked the value-added data for NYC teachers for several years, and we can look at those scores to [find out]. Here is one such graph [showing how the same teachers, in the same schools, scored in 2008-9 versus 2009-10]:

pic 8 - value added for 2 successive years Rubenstein NYC

That is very close to random.

How about teachers who teach the same subject to two different grade levels, say, fourth-grade math and fifth-grade math? Again, random points:

pic 9 - VAM for same subject different grades NYC rubenstein

One last point:

Mayor Gray and chancellors Henderson and Rhee all claim that education in DC only started improving after mayoral control of the schools, starting in 2007. Look for yourself [in the next two graphs].

pic 11 - naep 8th grade math avge scale scores since 1990 many states incl dc

 

pic 12 naep 4th grade reading scale scores since 1993 many states incl dc

Notice that gains began almost 20 years ago, long before mayoral control or chancellors Rhee and Henderson, long before IMPACT.

To repeat, I suggest that we throw out IMPACT and look hard at the ideas of Edwards Deming and the assessment models used in Montgomery County.

My Testimony Yesterday Before DC City Council’s Education Subcommittee ‘Roundtable’

Testimony of Guy Brandenburg, retired DCPS mathematics teacher before the DC City Council Committee on Education Roundtable, December 14, 2013 at McKinley Tech

 

Hello, Mr. Catania, audience members, and any other DC City Council members who may be present. I am a veteran DC math teacher who began teaching in Southeast DC about 35 years ago, and spent my last 15 years of teaching at Alice Deal JHS/MS. I taught everything from remedial 7th grade math through pre-calculus, as well as computer applications.

Among other things, I coached MathCounts teams at Deal and at Francis JHS, with my students often taking first place against all other public, private, and charter schools in the city and going on to compete against other state teams. As a result, I have several boxes full of trophies and some teaching awards.

Since retiring, I have been helping Math for America – DC (which is totally different from Teach for America) in training and mentoring new but highly skilled math teachers in DC public and charter schools; operating a blog that mostly concerns education; teaching astronomy and telescope making as an unpaid volunteer; and also tutoring [as a volunteer] students at the school closest to my house in Brookland, where my daughter attended kindergarten about 25 years ago.

But this testimony is not about me; as a result, I won’t read the previous paragraphs aloud.

My testimony is about how the public is being deceived with bogus statistics into thinking things are getting tremendously better under mayoral control of schools and under the chancellorships of Rhee and Henderson.

In particular, I want to show that the Value-Added measurements that are used to evaluate teachers are unreliable, invalid, and do not help teachers improve their methods of instruction. To the contrary: IVA measurements are driving a number of excellent, veteran teachers to resign or be fired from DCPS to go elsewhere.

I will try to show this mostly with graphs made by me and others, because in statistics, a good scatter plot is worth many a word or formula.

John Ewing, who is the president of Math for America and is a former executive director of the American Mathematical Society, wrote that VAM is “mathematical intimidation” and not reliable. I quote:

pic 1 john ewing

 

In case you were wondering how the formula goes, this is all that one of my colleagues was able to pry from Jason Kamras after SIX MONTHS of back-and-forth emails asking for additional information:

pic 2 dcps iva vam formula

One problem with that formula is that nobody outside a small group of highly-paid consultants has any idea what are the values of any of those variables. What’s more, many of those variables are composed of lists or matrices (“vectors”) of other variables.

In not a single case has the Office of Data and Accountability sat down with a teacher and explained, in detail, exactly how a teachers’ score is calculated, student by student, class by class, test score by test score.

Nor has that office shared that data with the Washington Teachers’ Union.

It’s the mathematics of intimidation, lack of accountability, and obfuscation.

I would ask you, Mr. Catania, to ask the Office of Data and Accountability to share with the WTU all IMPACT scores for every single teacher, including all the sub-scores, such as those for IVA and classroom observations.

To put a personal touch to my data, one of my former Deal colleagues shared with me that she resigned from DCPS specifically because her IVA scores kept bouncing around with no apparent reason. In fact, the year that she thought she did her very best job ever in her entire career – that’s when she earned her lowest value-added score. She now teaches in Montgomery County and recently earned the distinction of becoming a National Board Certified teacher – a loss for DCPS students, but a gain for those in Maryland.

Bill Turque of the Washington Post documented the case of Sarah Wysocki, an excellent teacher with outstanding classroom observation results, who was fired by DCPS for low IVA scores. She is now happily teaching in Virginia. I am positive that these two examples can be multiplied many times over.

Now let’s look at some statistics. As I mentioned, in many cases, pictures and graphs speak more clearly than words or numbers or equations.

My first graph is of completely random data points that should show absolutely no correlation with each other, meaning, they are not linked to each other in any way. I had my Excel spreadsheet to make two lists of random numbers, and I plotted those as the x- and y- variables on the following graph.

pic 3 - completely random points

I asked Excel also to draw a line of best fit and to calculate the correlation coefficient R and R-squared. It did so, as you can see, R-squared is very low, about 0.08 (eight percent). R, the square root of R-squared, is about 29 percent.

Remember, those are completely random numbers generated by Excel.

Now let’s look at a very strong correlation of real numbers: poverty rates and student achievement in a number of states. The first one is for Nebraska.

pic  4 - nebraska poverty vs achievement

R would be about 94% in this case – a very strong correlation indeed.

The next one is for Wisconsin:

pic 5 - wisconsin poverty vs achievement

Again, quite a strong correlation – a negative one: the poorer the student body, the lower the average achievement, which we see repeated in every state and every country in the world. Including DC, as you can see here:

 

pic 6 - poverty vs proficiency in DC

Now, how about those Value-Added scores? Do they correlate with classroom observations?

Mostly, we don’t know, because the data is kept secret. However, someone leaked to me the IVA and classroom observation scores for all DCPS teachers for SY 2009-10, and I plotted them. Is this a strong correlation, or not?

pic 7 - VAM versus TLF in DC IMPACT 2009-10

I would say this looks pretty much like no correlation at all. What on earth are these two things measuring? It certainly gives teachers no assistance on what to improve in order to help their students learn better.

And how stable are Value-Added measurements over time? If they are stable, that would mean that we might be able to use them to weed out the teachers who consistently score at the bottom, and reward those who consistently score at the top.

Unfortunately, since DCPS keeps all the data hidden, we don’t exactly know how stable these scores are here. However, the New York Times leaked the value-added data for NYC teachers for several years, and we can look at those scores to see.

Here is one such graph:

pic 8 - value added for 2 successive years Rubenstein NYC

That is very close to random.

How about teachers who teach the same subject to two different grade levels (say, fourth-grade math and fifth-grade math)? Again, random points:

pic 9 - VAM for same subject different grades NYC rubenstein

One thing that all veteran teachers agree on is that they stunk at their job during their first year and got a lot better their second year. This should show up on value-added graphs of year 1 versus year 2 scores for the same teachers, right?

Wrong.

Take a look:

pic 10 - VAM first yr vs second year same teacher rubenstein nyc

One last point:

Mayor Gray and chancellors Henderson and Rhee all claim that education in DC only started improving after mayoral control of the schools, starting in 2007.

Graphs and the NAEP show a different story. We won’t know until next week how DCPS and the charter schools did, separately, for 2013, but the following graphs show that reading andmath scores for DC fourth- and eighth-graders have been rising fairly steadily for nearly twenty years, or long before mayoral control or the appointments of our two chancellors (Rhee and Henderson).

 

 

pic 13 - naep reading 8th since 1998 scale scores many states incl dc

 

pic 12 naep 4th grade reading scale scores since 1993 many states incl dc

pic 11 - naep 8th grade math avge scale scores since 1990 many states incl dc

 

 

%d bloggers like this: