I had been also thinking (but did not write down) that the tide would turn, and that teachers and schools themselves would no longer be seen as the whole cause of poverty or brilliance.
Instead, I feared that racists would once again become free to loudly and publicly blame black and brown people for their own poverty, just like they did from the end of Reconstruction 147 years ago, right up through the anti-Civil Rights backlash of the 1960s and 1970s.
In 1983 the essay (disguised as an objective report) called “Nation At Risk” (NAR) jump-started an Education Reform drive that was truly bi-partisan, and had scores of billionaires both liberal (eg Gates) and conservative (eg Waltons) willing to fund it and push politicians to back it. Presidents Clinton, Bush2, and Obama and their secretaries of education all embraced it.
If you hadn’t noticed, this reform movement failed completely.
By its own terms (that is, test scores).
Despite having ‘edu-reformers’ in charge of every single large public school system in the nation.
But it took a while for those failures to become obvious.
At first, only a handful of writers such as Gerald Bracey pointed out the errors in that study and in the reformers’ steamroller. When the report first came out, I was teaching math to 7th graders in a very poor region of DC, and felt embarrassed that so many of my students there (100% Black) did so poorly in school, despite my efforts and those of my colleagues.
Some teachers from East Asia warned me not to believe the hype surrounding NAR. They said the model of education that exists in China, Japan, Korea and so on was NOT one that should be emulated by the US.
But I did believe the myth.
Later, I read some columns by Bracey and others and began to have doubts.
Then the amazing fraud Michelle Rhee was given control of DC’s entire public school system in 2007, less than two years before I retired.
I had never heard of the woman before, but upon her being named Chancellor of DCPS, I heard that she claimed to have performed educational miracles in a low-income, all-Black public school as a 3rd and 4th grade charter school teacher in Baltimore. She wrote in her resume: “Over a two-year period” in the mid-1990s she “moved students scoring on average at the 13th percentile on national standardized tests to 90 percent of students scoring at the 90th percentile or higher.”
When I read that sentence in herresume, I seem to recall my jaw literally dropping open.
If you have ever been around kids and looked at their test scores, you would realize that this feat would be the equivalent of landing a triple axel in ice skating, while also sinking a three-pointer in the NBA, and running a marathon in under two hours.
Simultaneously.
If this really had happened, it would have been front-page news in every single publication that dealt with education.
(Sounds like George Santos took lessons on fake resume claims from Michelle Rhee!)
Of course, there were no such articles. So I scratched my head and wondered.
After I retired, someone pointed me to where the fairly detailed Baltimore test scores could be found. I looked at them, and found that she had mythologized a small bump in test scores into the greatest educational achievement ever accomplished, anywhere. And nobody had called her on this lie.
I suspect that the bump can be attributed in large part to the fact that over one third of the students at her grade level, at that school, in that year, had scores that were so low that they weren’t counted!!! I wrote a few posts on my blog about it, and even did a call-in on an NPR interview with her, asking why she lied so much, in particular about those scores. She just giggled, as if to imply that I was just being an idiot for trying to call her on such a small technicality, when she was still working miracles.
====
From theDaily Howler: “In the 1994-1995 school year, the seven schools run by EAI were under enormous pressure. During and after the previous year, major disputes had broken out about the low test scores of the EAI schools; by the fall of 1994, everyone knew that the pressure was on, that the plug might be pulled on the program. (As a simple Nexis search will show, all these matters were being discussed in the Baltimore Sun.) Do we possess three brain cells among us? If any school in the EAI group had an educational miracle occurring, this glorious fact would have been shouted to the skies by EAI’s corporate leadership. Trust us: The teachers involved would have gained acclaim in the national media—the kind of “acclaim” Rhee used to say she had attained, before she realized she had to stop saying it. It’s absurd to think there was some large group of third-graders “scoring at the 90th percentile or higher,” but their test scores somehow never came to the attention of the UMBC researchers.”
=====
Apparently nobody else with any knowledge of basic, elementary statistics and probability had previously bothered to compare those actual scores with her extraordinary claims. So I wrote what I found, with a fair amount of fury at the fact that such an amazing, world-class fraud and liar could be in charge of education in my home city, Washington, DC, the very seat of national government and so on. I got my 15 minutes of fame, but while Rhee did retire in disgrace, she has unfortunately never been indicted for fraud, even though she clearlysuborned all sorts of cheating and erasing of bubble marks on students’ tests, and gave prizes and awards to one of the most prolific cheaters, a principal in my own neighborhood. (see here for some details.)
As I have recently feared, but did not put into writing, the really scary part now is that right-wingers and racists are using the failure of this billionaire-led disruption of public schools to get rid of the very idea of public education as a public good. They applaud the self-segregation.
The ‘Value-Added Measurement’ movement in American education, implemented in part by the now-disgraced Michelle Rhee here in DC, has been a complete and utter failure, even as measured by its own yardsticks, as you will see below
Yet, the same corporate ‘reformers’ who were its major cheerleaders do not conclude from this that the idea was a bad one. Instead, they claim that it wasn’t tried with enough rigor and fidelity.
How to Learn Nothing from the Failure of VAM-Based Teacher Evaluation
The Annenberg Institute for School Reform is a most exclusive academic club lavishly funded and outfitted at Brown U. for the advancement of corporate education in America.
The Institute is headed by Susanna Loeb, who has a whole slew of degrees from prestigious universities, none of which has anything to do with the science and art of schooling, teaching, or learning.
Researchers at the Institute are circulating aworking paper that, at first glance, would suggest that school reformers might have learned something about the failure of teacher evaluation based on value-added models applied to student test scores. The abstract:
Starting in 2009, the U.S. public education system undertook a massive effort to institute new high-stakes teacher evaluation systems. We examine the effects of these reforms on student achievement and attainment at a national scale by exploiting the staggered timing of implementation across states. We find precisely estimated null effects, on average, that rule out impacts as small as 1.5 percent of a standard deviation for achievement and 1 percentage point for high school graduation and college enrollment. We also find little evidence of heterogeneous effects across an index measuring system design rigor, specific design features, and district characteristics. [my emphasis – GFB]
So could this mean that the national failure of VAM applied to teacher evaluation might translate to decreasing the brutalization of teachers and the waste of student learning time that resulted from the implementation of VAM beginning in 2009?
No such luck.
The conclusion of the paper, in fact, clearly shows that the Annenbergers have concluded that the failure to raise test scores by corporate accountability means (VAM) resulted from laggard states and districts that did not adhere strictly to the VAM’s mad methods. In short, the corporate-led failure of VAM in education happened as a result of schools not being corporate enough:
Firms in the private sector often fail to implement best management practices and performance evaluation systems because of imperfectly competitive markets and the costs of implementing such policies and practices (Bloom and Van Reenen 2007). These same factors are likely to have influenced the design and implementation of teacher evaluation reforms. Unlike firms in a perfectly competitive market with incentives to implement management and evaluation systems that increase productivity, school districts and states face less competitive pressure to innovate. Similarly, adopting evaluation systems like the one implemented in Washington D.C. requires a significant investment of time, money, and political capital. Many states may have believed that the costs of these investments outweighed the benefits. Consequently, the evaluation systems adopted by many states were not meaningfully different from the status quo and subsequently failed to improve student outcomes.
So the Gates-Duncan RTTT corporate plan for teacher evaluation failed not because it was a corporate model but because it was not corporate enough! In short, there were way too many small carrots and not enough big sticks.
This is a long article printed on Alternet. A teacher believed so much all the hype about magical charter schools that she tried twice to form her own charter school. (She failed to get approval, rightly so, she says.)
So she went to work for an existing charter school (whose name she doesn’t provide) in Los Angeles and the scales fell from her eyes.
[Ed. Note: As DC’s office of the state superintendent of education (OSSE) seeks a waiver of PARCC testing again (recall that OSSE waived PARCC last school year due to the pandemic) and the DC auditor just released a bombshell report of poor stewardship of DC’s education data, it is time to revisit how standardized test data, teacher evaluations, and harsh school penalties were united by ed reformers in DCPS under mayoral control. This first-hand account of what went down in DCPS, the first of two parts by semi-retired educator Richard P. Phelps, appeared in Nonpartisan Education Review in September 2020 and is reprinted here with permission. The author thanks DC budget expert Mary Levy and retired DCPS teacher Erich Martel for their helpful comments in the research of this article.]
By Richard P. Phelps
Ten years ago, I worked as the director of assessments for DCPS. My tenure coincided with Michelle Rhee’s last nine months as chancellor. I departed shortly after Vincent Gray defeated Adrian Fenty in the September 2010 DC mayoral primary.
My primary task was to design an expansion of that testing program that served the IMPACT teacher evaluation system to include all core subjects and all grade levels. Despite its fame (or infamy), the test score aspect of the IMPACT program affected only 13% of teachers, those teaching either reading or math in grades four through eight. Only those subjects and grade levels included the requisite pre- and post-tests required for teacher “value added” measurements (VAM). Not included were most subjects (e.g., science, social studies, art, music, physical education), grades kindergarten to two, and high school.
Chancellor Rhee wanted many more teachers included. So, I designed a system that would cover more than half the DCPS teacher force, from kindergarten through high school. You haven’t heard about it because it never happened. The newly elected Vincent Gray had promised during his mayoral campaign to reduce the amount of testing; the proposed expansion would have increased it fourfold.
VAM affected teachers’ jobs. A low value-added score could lead to termination; a high score, to promotion and a cash bonus. VAM as it was then structured was obviously, glaringly flawed, as anyone with a strong background in educational testing could have seen. Unfortunately, among the many new central office hires from the elite of ed reform circles, none had such a background. (Even a primary grades teacher with the same group of students the entire school day had those students for less than six hours a day, five days a week, for less than half the year. All told, even in the highest exposure circumstances, a teacher interacted with the same group of students for less than a tenth of each student’s waking hours in a year, and for less than a twentieth in the tested subjects of English and math. In the lowest exposure circumstance, a high school teacher might interact with a class of English or math students for less than three percent of a student’s annual hours.
Before posting a request for proposals from commercial test developers for the testing expansion plan, I was instructed to survey two groups of stakeholders—central office managers and school-level teachers and administrators.
Not surprisingly, some of the central office managers consulted requested additions or changes to the proposed testing program where they thought it would benefit their domain of responsibility. The net effect on school-level personnel would have been to add to their administrative burden. Nonetheless, all requests from central office managers would be honored.
The Grand Tour
At about the same time, over several weeks of the late spring and early summer of 2010, along with a bright summer intern, I visited a dozen DCPS schools. The alleged purpose was to collect feedback on the design of the expanded testing program. I enjoyed these meetings. They were informative, animated, and very well attended. School staff appreciated the apparent opportunity to contribute to policy decisions and tried to make the most of it.
Each school greeted us with a full complement of faculty and staff on their days off, numbering a several dozen educators at some venues. They believed what we had told them: that we were in the process of redesigning the DCPS assessment program and were genuinely interested in their suggestions for how best to do it.
At no venue did we encounter stand-pat knee-jerk rejection of education reform efforts. Some educators were avowed advocates for the Rhee administration’s reform policies, but most were basically dedicated educators determined to do what was best for their community within the current context.
The Grand Tour was insightful, too. I learned for the first time of certain aspects of DCPS’s assessment system that were essential to consider in its proper design, aspects of which the higher-ups in the DCPS Central Office either were not aware or did not consider relevant.
The group of visited schools represented DCPS as a whole in appropriate proportions geographically, ethnically, and by education level (i.e., primary, middle, and high). Within those parameters, however, only schools with “friendly” administrations were chosen. That is, we only visited schools with principals and staff openly supportive of the Rhee-Henderson agenda.
But even they desired changes to the testing program, whether or not it was expanded. Their suggestions covered both the annual districtwide DC-CAS (or “comprehensive” assessment system), on which the teacher evaluation system was based, and the DC-BAS (or “benchmarking” assessment system), a series of four annual “no-stakes” interim tests unique to DCPS, ostensibly offered to help prepare students and teachers for the consequential-for-some-school-staff DC-CAS. (Though officially “no stakes,” some principals analyzed results from the DC-BAS to identify students whose scores lay just under the next higher benchmark and encouraged teachers to focus their instructional efforts on them. Moreover, at the high school level, where testing occurred only in grade 10, students who performed poorly on the DC-BAS might be artificially re-classified as held-back 9th graders or advanced prematurely to 11th grade in order to avoid the DC-CAS.)
At each staff meeting I asked for a show of hands on several issues of interest that I thought were actionable. Some suggestions for program changes received close to unanimous support. Allow me to describe several.
***Move DC-CAS test administration later in the school year. Many citizens may have logically assumed that the IMPACT teacher evaluation numbers were calculated from a standard pre-post test schedule, testing a teacher’s students at the beginning of their academic year together and then again at the end. In 2010, however, the DC-CAS was administered in March, three months before school year end. Moreover, that single administration of the test served as both pre- and post-test, posttest for the current school year and pretest for the following school year. Thus, before a teacher even met their new students in late August or early September, almost half of the year for which teachers were judged had already transpired—the three months in the spring spent with the previous year’s teacher and almost three months of summer vacation.
School staff recommended pushing DC-CAS administration to later in the school year. Furthermore, they advocated a genuine pre-post-test administration schedule—pre-test the students in late August–early September and post-test them in late-May–early June—to cover a teacher’s actual span of time with the students.
This suggestion was rejected because the test development firm with the DC-CAS contract required three months to score some portions of the test in time for the IMPACT teacher ratings scheduled for early July delivery, before the start of the new school year. Some small number of teachers would be terminated based on their IMPACT scores, so management demanded those scores be available before preparations for the new school year began. (Even a primary grades teacher with the same group of students the entire school day had those students for less than six hours a day, five days a week, for less than half the year. All told, even in the highest exposure circumstances, a teacher interacted with the same group of students for less than a tenth of each student’s waking hours in a year, and for less than a twentieth in the tested subjects of English and math. In the lowest exposure circumstance, a high school teacher might interact with a class of English or math students for less than three percent of a student’s annual hours.)
The tail wagged the dog.
***Add some stakes to the DC-CAS in the upper grades. Because DC-CAS test scores portended consequences for teachers but none for students, some students expended little effort on the test. Indeed, extensive research on “no-stakes” (for students) tests reveal that motivation and effort vary by a range of factors including gender, ethnicity, socioeconomic class, the weather, and age. Generally, the older the student, the lower the test-taking effort. This disadvantaged some teachers in the IMPACT ratings for circumstances beyond their control: unlucky student demographics.
Central office management rejected this suggestion to add even modest stakes to the upper grades’ DC-CAS; no reason given.
***Move one of the DC-BAS tests to year end. If management rejected the suggestion to move DC-CAS test administration to the end of the school year, school staff suggested scheduling one of the no-stakes DC-BAS benchmarking tests for late May–early June. As it was, the schedule squeezed all four benchmarking test administrations between early September and mid-February. Moving just one of them to the end of the year would give the following year’s teachers a more recent reading (by more than three months) of their new students’ academic levels and needs.
Central office management rejected this suggestion probably because the real purpose of the DC-BAS was not to help teachers understand their students’ academic levels and needs, as the following will explain.
***Change DC-BAS tests so they cover recently taught content. Many DC citizens probably assumed that, like most tests, the DC-BAS interim tests covered recently taught content, such as that covered since the previous test administration. Not so in 2010. The first annual DC-BAS was administered in early September, just after the year’s courses commenced. Moreover, it covered the same content domain—that for the entirety of the school year—as each of the next three DC-BAS tests.
School staff proposed changing the full-year “comprehensive” content coverage of each DC-BAS test to partial-year “cumulative” coverage, so students would only be tested on what they had been taught prior to each test administration.
This suggestion, too, was rejected. Testing the same full-year comprehensive content domain produced a predictable, flattering score rise. With each DC-BAS test administration, students recognized more of the content, because they had just been exposed to more of it, so average scores predictably rose. With test scores always rising, it looked like student achievement improved steadily each year. Achieving this contrived score increase required testing students on some material to which they had not yet been exposed, both a violation of professional testing standards and a poor method for instilling student confidence. (Of course, it was also less expensive to administer essentially the same test four times a year than to develop four genuinely different tests.)
***Synchronize the sequencing of curricular content across the District. DCPS management rhetoric circa 2010 attributed classroom-level benefits to the testing program. Teachers would know more about their students’ levels and needs and could also learn from each other. Yet, the only student test results teachers received at the beginning of each school year was half-a-year old, and most of the information they received over the course of four DC-BAS test administrations was based on not-yet-taught content.
As for cross-district teacher cooperation, unfortunately there was no cross-District coordination of common curricular sequences. Each teacher paced their subject matter however they wished and varied topical emphases according to their own personal preference.
It took DCPS’s chief academic officer, Carey Wright, and her chief of staff, Dan Gordon, less than a minute to reject the suggestion to standardize topical sequencing across schools so that teachers could consult with one another in real time. Tallying up the votes: several hundred school-level District educators favored the proposal, two of Rhee’s trusted lieutenants opposed it. It lost.
***Offer and require a keyboarding course in the early grades. DCPS was planning to convert all its testing from paper-and-pencil mode to computer delivery within a few years. Yet, keyboarding courses were rare in the early grades. Obviously, without systemwide keyboarding training in computer use some students would be at a disadvantage in computer testing.
Suggestion rejected.
In all, I had polled over 500 DCPS school staff. Not only were all of their suggestions reasonable, some were essential in order to comply with professional assessment standards and ethics.
Nonetheless, back at DCPS’s central office, each suggestion was rejected without, to my observation, any serious consideration. The rejecters included Chancellor Rhee, the head of the office of data and accountability—the self-titled “Data Lady,” Erin McGoldrick—and the head of the curriculum and instruction division, Carey Wright, and her chief deputy, Dan Gordon.
Four central office staff outvoted several hundred school staff (and my recommendations as assessment director). In each case, the changes recommended would have meant some additional work on their parts, but in return for substantial improvements in the testing program. Their rhetoric was all about helping teachers and students; but the facts were that the testing program wasn’t structured to help them.
What was the purpose of my several weeks of school visits and staff polling? To solicit “buy in” from school level staff, not feedback.
Ultimately, the new testing program proposal would incorporate all the new features requested by senior central office staff, no matter how burdensome, and not a single feature requested by several hundred supportive school-level staff, no matter how helpful. Like many others, I had hoped that the education reform intention of the Rhee-Henderson years was genuine. DCPS could certainly have benefitted from some genuine reform.
Alas, much of the activity labelled “reform” was just for show, and for padding resumes. Numerous central office managers would later work for the Bill and Melinda Gates Foundation. Numerous others would work for entities supported by the Gates or aligned foundations, or in jurisdictions such as Louisiana, where ed reformers held political power. Most would be well paid.
Their genuine accomplishments, or lack thereof, while at DCPS seemed to matter little. What mattered was the appearance of accomplishment and, above all, loyalty to the group. That loyalty required going along to get along: complicity in maintaining the façade of success while withholding any public criticism of or disagreement with other in-group members.
Unfortunately, in the United States what is commonly showcased as education reform is neither a civic enterprise nor a popular movement. Neither parents, the public, nor school-level educators have any direct influence. Rather, at the national level, U.S. education reform is an elite, private club—a small group of tightly connected politicos and academics—a mutual admiration society dedicated to the career advancement, political influence, and financial benefit of its members, supported by a gaggle of wealthy foundations (e.g., Gates, Walton, Broad, Wallace, Hewlett, Smith-Richardson).
For over a decade, The Ed Reform Club exploited DC for its own benefit. Local elite formed the DC Public Education Fund (DCPEF) to sponsor education projects, such as IMPACT, which they deemed worthy. In the negotiations between the Washington Teachers’ Union and DCPS concluded in 2010, DCPEF arranged a 3-year grant of $64.5 million from the Arnold, Broad, Robertson, and Walton foundations to fund a 5-year retroactive teacher pay raise in return for contract language allowing teacher excessing tied to IMPACT, which Rhee promised would lead to annual student test score increases by 2012. Projected goals were not met; foundation support continued nonetheless.
Michelle Johnson (nee Rhee) chaired the board of a charter school chain in California and occasionally collects $30,000+ in speaker fees but, otherwise, seems to have deliberately withdrawn from the limelight. Despite contributing her own additional scandals after she assumed the DCPS chancellorship, Kaya Henderson ascended to great fame and glory with a “distinguished professorship” at Georgetown; honorary degrees from Georgetown and Catholic universities; gigs with the Chan Zuckerberg Initiative, Broad Leadership Academy, and Teach for All; and board memberships with The Aspen Institute, The College Board, Robin Hood NYC, and Teach For America. Carey Wright is now state superintendent in Mississippi. Dan Gordon runs a 30-person consulting firm, Education Counsel, which strategically partners with major players in U.S. education policy. The manager of the IMPACT teacher evaluation program, Jason Kamras, now works as superintendent of the Richmond, VA public schools.
Arguably the person most directly responsible for the recurring assessment system fiascos of the Rhee-Henderson years, then chief of data and accountability Erin McGoldrick, now specializes in “data innovation” as partner and chief operating officer at an education management consulting firm. Her firm, Kitamba, strategically partners with its own panoply of major players in U.S. education policy. Its list of recent clients includes the DC Public Charter School Board and DCPS.
If the ambitious DC central office folk who gaudily declared themselves leading education reformers were not really, who were the genuine education reformers during the Rhee-Henderson decade of massive upheaval and per-student expenditures three times those in the state of Utah? They were the school principals and staff whose practical suggestions were ignored by central office glitterati. They were whistleblowers like history teacher Erich Martel, who had documented DCPS’s manipulation of student records and phony graduation rates years before the investigation of Ballou High School and was demoted and then “excessed” by Henderson. Or school principal Adell Cothorne, who spilled the beans on test answer sheet “erasure parties” at Noyes Education Campus and lost her job under Rhee.
Ten years ago, I worked as the Director of Assessments for the District of Columbia Public Schools (DCPS). For temporal context, I arrived after the first of the infamous test cheating scandals and left just before the incident that spawned a second. Indeed, I filled a new position created to both manage test security and design an expanded testing program. I departed shortly after Vincent Gray, who opposed an expanded testing program, defeated Adrian Fenty in the September 2010 DC mayoral primary. My tenure coincided with Michelle Rhee’s last nine months as Chancellor.
The recurring test cheating scandals of the Rhee-Henderson years may seem extraordinary but, in fairness, DCPS was more likely than the average US school district to be caught because it received a much higher degree of scrutiny. Given how tests are typically administered in this country, the incidence of cheating is likely far greater than news accounts suggest, for several reasons:
· in most cases, those who administer tests—schoolteachers and administrators—have an interest in their results;
· test security protocols are numerous and complicated yet, nonetheless, the responsibility of non-expert ordinary school personnel, guaranteeing their inconsistent application across schools and over time;
· after-the-fact statistical analyses are not legal proof—the odds of a certain amount of wrong-to-right erasures in a single classroom on a paper-and-pencil test being coincidental may be a thousand to one, but one-in-a-thousand is still legally plausible; and
· after-the-fact investigations based on interviews are time-consuming, scattershot, and uneven.
Still, there were measures that the Rhee-Henderson administrations could have adopted to substantially reduce the incidence of cheating, but they chose none that might have been effective. Rather, they dug in their heels, insisted that only a few schools had issues, which they thoroughly resolved, and repeatedly denied any systematic problem.
Cheating scandals
From 2007 to 2009 rumors percolated of an extraordinary level of wrong-to-right erasures on the test answer sheets at many DCPS schools. “Erasure analysis” is one among several “red flag” indicators that testing contractors calculate to monitor cheating. The testing companies take no responsibility for investigating suspected test cheating, however; that is the customer’s, the local or state education agency.
In her autobiographical account of her time as DCPS Chancellor, Michelle Johnson (nee Rhee), wrote (p. 197)
“For the first time in the history of DCPS, we brought in an outside expert to examine and audit our system. Caveon Test Security – the leading expert in the field at the time – assessed our tests, results, and security measures. Their investigators interviewed teachers, principals, and administrators.
“Caveon found no evidence of systematic cheating. None.”
Caveon, however, had not looked for “systematic” cheating. All they did was interview a few people at several schools where the statistical anomalies were more extraordinary than at others. As none of those individuals would admit to knowingly cheating, Caveon branded all their excuses as “plausible” explanations. That’s it; that is all that Caveon did. But, Caveon’s statement that they found no evidence of “widespread” cheating—despite not having looked for it—would be frequently invoked by DCPS leaders over the next several years.[1]
Incidentally, prior to the revelation of its infamous decades-long, systematic test cheating, the Atlanta Public Schools had similarly retained Caveon Test Security and was, likewise, granted a clean bill of health. Only later did the Georgia state attorney general swoop in and reveal the truth.
In its defense, Caveon would note that several cheating prevention measures it had recommended to DCPS were never adopted.[2] None of the cheating prevention measures that I recommended were adopted, either.
The single most effective means for reducing in-classroom cheating would have been to rotate teachers on test days so that no teacher administered a test to his or her own students. It would not have been that difficult to randomly assign teachers to different classrooms on test days.
The single most effective means for reducing school administratorcheating would have been to rotate test administrators on test days so that none managed the test materials for their own schools. The visiting test administrators would have been responsible for keeping test materials away from the school until test day, distributing sealed test booklets to the rotated teachers on test day, and for collecting re-sealed test booklets at the end of testing and immediately removing them from the school.
Instead of implementing these, or a number of other feasible and effective test security measures, DCPS leaders increased the number of test proctors, assigning each of a few dozen or so central office staff a school to monitor. Those proctors could not reasonably manage the volume of oversight required. A single DC test administration could encompass a hundred schools and a thousand classrooms.
Investigations
So, what effort, if any, did DCPS make to counter test cheating? They hired me, but then rejected all my suggestions for increasing security. Also, they established a telephone tip line. Anyone who suspected cheating could report it, even anonymously, and, allegedly, their tip would be investigated.
Some forms of cheating are best investigated through interviews. Probably the most frequent forms of cheating at DCPS—teachers helping students during test administrations and school administrators looking at test forms prior to administration—leave no statistical residue. Eyewitness testimony is the only type of legal evidence available in such cases, but it is not just inconsistent, it may be socially destructive.
I remember two investigations best: one occurred in a relatively well-to-do neighborhood with well-educated parents active in school affairs; the other in one of the city’s poorest neighborhoods. Superficially, the cases were similar—an individual teacher was accused of helping his or her own students with answers during test administrations. Making a case against either elementary school teacher required sworn testimony from eyewitnesses, that is, students—eight-to-ten-year olds.
My investigations, then, consisted of calling children into the principal’s office one-by-one to be questioned about their teacher’s behavior. We couldn’t hide the reason we were asking the questions. And, even though each student agreed not to tell others what had occurred in their visit to the principal’s office, we knew we had only one shot at an uncorrupted jury pool.
Though the accusations against the two teachers were similar and the cases against them equally strong, the outcomes could not have been more different. In the high-poverty neighborhood, the students seemed suspicious and said little; none would implicate the teacher, whom they all seemed to like.
In the more prosperous neighborhood, students were more outgoing, freely divulging what they had witnessed. The students had discussed the alleged coaching with their parents who, in turn, urged them to tell investigators what they knew. During his turn in the principal’s office, the accused teacher denied any wrongdoing. I wrote up each interview, then requested that each student read and sign.
Thankfully, that accused teacher made a deal and left the school system a few weeks later. Had he not, we would have required the presence in court of the eight-to-ten-year olds to testify under oath against their former teacher, who taught multi-grade classes. Had that prosecution not succeeded, the eyewitness students could have been routinely assigned to his classroom the following school year.
My conclusion? Only in certain schools is the successful prosecution of a cheating teacher through eyewitness testimony even possible. But, even where possible, it consumes inordinate amounts of time and, otherwise, comes at a high price, turning young innocents against authority figures they naturally trusted.
Cheating blueprints
Arguably the most widespread and persistent testing malfeasance in DCPS received little attention from the press. Moreover, it was directly propagated by District leaders, who published test blueprints on the web. Put simply, test “blueprints” are lists of the curricular standards (e.g., “student shall correctly add two-digit numbers”) and the number of test items included in an upcoming test related to each standard. DC had been advance publishing its blueprints for years.
I argued that the way DC did it was unethical. The head of the Division of Data & Accountability, Erin McGoldrick, however, defended the practice, claimed it was common, and cited its existence in the state of California as precedent. The next time she and I met for a conference call with one of DCPS’s test providers, Discover Education, I asked their sales agent how many of their hundreds of other customers advance-published blueprints. His answer: none.
In the state of California, the location of McGoldrick’s only prior professional experience, blueprints were, indeed, published in advance of test administrations. But their tests were longer than DC’s and all standards were tested. Publication of California’s blueprints served more to remind the populace what the standards were in advance of each test administration. Occasionally, a standard considered to be of unusual importance might be assigned a greater number of test items than the average, and the California blueprints signaled that emphasis.
In Washington, DC, the tests used in judging teacher performance were shorter, covering only some of each year’s standards. So, DC’s blueprints showed everyone well in advance of the test dates exactly which standards would be tested and which would not. For each teacher, this posed an ethical dilemma: should they “narrow the curriculum” by teaching only that content they knew would be tested? Or, should they do the right thing and teach all the standards, as they were legally and ethically bound to, even though it meant spending less time on the to-be-tested content? It’s quite a conundrum when one risks punishment for behaving ethically.
Monthly meetings convened to discuss issues with the districtwide testing program, the DC Comprehensive Assessment System (DC-CAS)—administered to comply with the federal No Child Left Behind (NCLB) Act. All public schools, both DCPS and charters, administered those tests. At one of these regular meetings, two representatives from the Office of the State Superintendent of Education (OSSE) announced plans to repair the broken blueprint process.[3]
The State Office employees argued thoughtfully and reasonably that it was professionally unethical to advance publish DC test blueprints. Moreover, they had surveyed other US jurisdictions in an effort to find others that followed DC’s practice and found none. I was the highest-ranking DCPS employee at the meeting and I expressed my support, congratulating them for doing the right thing. I assumed that their decision was final.
I mentioned the decision to McGoldrick, who expressed surprise and speculation that it might have not been made at the highest level in the organizational hierarchy. Wasting no time, she met with other DCPS senior managers and the proposed change was forthwith shelved. In that, and other ways, the DCPS tail wagged the OSSE dog.
* * *
It may be too easy to finger ethical deficits for the recalcitrant attitude toward test security of the Rhee-Henderson era ed reformers. The columnist Peter Greene insists that knowledge deficits among self-appointed education reformers also matter:
“… the reformistan bubble … has been built from Day One without any actual educators inside it. Instead, the bubble is populated by rich people, people who want rich people’s money, people who think they have great ideas about education, and even people who sincerely want to make education better. The bubble does not include people who can turn to an Arne Duncan or a Betsy DeVos or a Bill Gates and say, ‘Based on my years of experience in a classroom, I’d have to say that idea is ridiculous bullshit.’”
“There are a tiny handful of people within the bubble who will occasionally act as bullshit detectors, but they are not enough. The ed reform movement has gathered power and money and set up a parallel education system even as it has managed to capture leadership roles within public education, but the ed reform movement still lacks what it has always lacked–actual teachers and experienced educators who know what the hell they’re talking about.”
In my twenties, I worked for several years in the research department of a state education agency. My primary political lesson from that experience, consistently reinforced subsequently, is that most education bureaucrats tell the public that the system they manage works just fine, no matter what the reality. They can get away with this because they control most of the evidence and can suppress it or spin it to their advantage.
In this proclivity, the DCPS central office leaders of the Rhee-Henderson era proved themselves to be no different than the traditional public-school educators they so casually demonized.
US school systems are structured to be opaque and, it seems, both educators and testing contractors like it that way. For their part, and contrary to their rhetoric, Rhee, Henderson, and McGoldrick passed on many opportunities to make their system more transparent and accountable.
[1] A perusal of Caveon’s website clarifies that their mission is to help their clients–state and local education departments–not get caught. Sometimes this means not cheating in the first place; other times it might mean something else. One might argue that, ironically, Caveon could be helping its clients to cheat in more sophisticated ways and cover their tracks better.
[2] Among them: test booklets should be sealed until the students open them and resealed by the students immediately after; and students should be assigned seats on test day and a seating chart submitted to test coordinators (necessary for verifying cluster patterns in student responses that would suggest answer copying).
[3] Yes, for those new to the area, the District of Columbia has an Office of the “State” Superintendent of Education (OSSE). Its domain of relationships includes not just the regular public schools (i.e., DCPS), but also other public schools (i.e., charters) and private schools. Practically, it primarily serves as a conduit for funneling money from a menagerie of federal education-related grant and aid programs
Short answer: nothing that would actually help students or teachers. But it’s made for well-padded resumes for a handful of insiders.
This is an important review, by the then-director of assessment. His criticisms echo the points that I have been making along with Mary Levy, Erich Martel, Adell Cothorne, and many others.
Looking Back on DC Education Reform 10 Years After,
Part 1: The Grand Tour
Richard P Phelps
Ten years ago, I worked as the Director of Assessments for the District of Columbia Public Schools (DCPS). My tenure coincided with Michelle Rhee’s last nine months as Chancellor. I departed shortly after Vincent Gray defeated Adrian Fenty in the September 2010 DC mayoral primary.
My primary task was to design an expansion of that testing program that served the IMPACT teacher evaluation system to include all core subjects and all grade levels. Despite its fame (or infamy), the test score aspect of the IMPACT program affected only 13% of teachers, those teaching either reading or math in grades four through eight. Only those subjects and grade levels included the requisite pre- and post-tests required for teacher “value added” measurements (VAM). Not included were most subjects (e.g., science, social studies, art, music, physical education), grades kindergarten to two, and high school.
Chancellor Rhee wanted many more teachers included. So, I designed a system that would cover more than half the DCPS teacher force, from kindergarten through high school. You haven’t heard about it because it never happened. The newly elected Vincent Gray had promised during his mayoral campaign to reduce the amount of testing; the proposed expansion would have increased it fourfold.
VAM affected teachers’ jobs. A low value-added score could lead to termination; a high score, to promotion and a cash bonus. VAM as it was then structured was obviously, glaringly flawed,[1] as anyone with a strong background in educational testing could have seen. Unfortunately, among the many new central office hires from the elite of ed reform circles, none had such a background.
Before posting a request for proposals from commercial test developers for the testing expansion plan, I was instructed to survey two groups of stakeholders—central office managers and school-level teachers and administrators.
Not surprisingly, some of the central office managers consulted requested additions or changes to the proposed testing program where they thought it would benefit their domain of responsibility. The net effect on school-level personnel would have been to add to their administrative burden. Nonetheless, all requests from central office managers would be honored.
The Grand Tour
At about the same time, over several weeks of the late Spring and early Summer of 2010, along with a bright summer intern, I visited a dozen DCPS schools. The alleged purpose was to collect feedback on the design of the expanded testing program. I enjoyed these meetings. They were informative, animated, and very well attended. School staff appreciated the apparent opportunity to contribute to policy decisions and tried to make the most of it.
Each school greeted us with a full complement of faculty and staff on their days off, numbering a several dozen educators at some venues. They believed what we had told them: that we were in the process of redesigning the DCPS assessment program and were genuinely interested in their suggestions for how best to do it.
At no venue did we encounter stand-pat knee-jerk rejection of education reform efforts. Some educators were avowed advocates for the Rhee administration’s reform policies, but most were basically dedicated educators determined to do what was best for their community within the current context.
The Grand Tour was insightful, too. I learned for the first time of certain aspects of DCPS’s assessment system that were essential to consider in its proper design, aspects of which the higher-ups in the DCPS Central Office either were not aware or did not consider relevant.
The group of visited schools represented DCPS as a whole in appropriate proportions geographically, ethnically, and by education level (i.e., primary, middle, and high). Within those parameters, however, only schools with “friendly” administrations were chosen. That is, we only visited schools with principals and staff openly supportive of the Rhee-Henderson agenda.
But even they desired changes to the testing program, whether or not it was expanded. Their suggestions covered both the annual districtwide DC-CAS (or “comprehensive” assessment system), on which the teacher evaluation system was based, and the DC-BAS (or “benchmarking” assessment system), a series of four annual “no-stakes” interim tests unique to DCPS, ostensibly offered to help prepare students and teachers for the consequential-for-some-school-staff DC-CAS.[2]
At each staff meeting I asked for a show of hands on several issues of interest that I thought were actionable. Some suggestions for program changes received close to unanimous support. Allow me to describe several.
1.Move DC-CAS test administration later in the school year. Many citizens may have logically assumed that the IMPACT teacher evaluation numbers were calculated from a standard pre-post test schedule, testing a teacher’s students at the beginning of their academic year together and then again at the end. In 2010, however, the DC-CAS was administered in March, three months before school year end. Moreover, that single administration of the test served as both pre- and post-test, posttest for the current school year and pretest for the following school year. Thus, before a teacher even met their new students in late August or early September, almost half of the year for which teachers were judged had already transpired—the three months in the Spring spent with the previous year’s teacher and almost three months of summer vacation.
School staff recommended pushing DC-CAS administration to later in the school year. Furthermore, they advocated a genuine pre-post-test administration schedule—pre-test the students in late August–early September and post-test them in late-May–early June—to cover a teacher’s actual span of time with the students.
This suggestion was rejected because the test development firm with the DC-CAS contract required three months to score some portions of the test in time for the IMPACT teacher ratings scheduled for early July delivery, before the start of the new school year. Some small number of teachers would be terminated based on their IMPACT scores, so management demanded those scores be available before preparations for the new school year began.[3] The tail wagged the dog.
2.Add some stakes to the DC-CAS in the upper grades. Because DC-CAS test scores portended consequences for teachers but none for students, some students expended little effort on the test. Indeed, extensive research on “no-stakes” (for students) tests reveal that motivation and effort vary by a range of factors including gender, ethnicity, socioeconomic class, the weather, and age. Generally, the older the student, the lower the test-taking effort. This disadvantaged some teachers in the IMPACT ratings for circumstances beyond their control: unlucky student demographics.
Central office management rejected this suggestion to add even modest stakes to the upper grades’ DC-CAS; no reason given.
3.Move one of the DC-BAS tests to year end. If management rejected the suggestion to move DC-CAS test administration to the end of the school year, school staff suggested scheduling one of the no-stakes DC-BAS benchmarking tests for late May–early June. As it was, the schedule squeezed all four benchmarking test administrations between early September and mid-February. Moving just one of them to the end of the year would give the following year’s teachers a more recent reading (by more than three months) of their new students’ academic levels and needs.
Central Office management rejected this suggestion probably because the real purpose of the DC-BAS was not to help teachers understand their students’ academic levels and needs, as the following will explain.
4.Change DC-BAS tests so they cover recently taught content. Many DC citizens probably assumed that, like most tests, the DC-BAS interim tests covered recently taught content, such as that covered since the previous test administration. Not so in 2010. The first annual DC-BAS was administered in early September, just after the year’s courses commenced. Moreover, it covered the same content domain—that for the entirety of the school year—as each of the next three DC-BAS tests.
School staff proposed changing the full-year “comprehensive” content coverage of each DC-BAS test to partial-year “cumulative” coverage, so students would only be tested on what they had been taught prior to each test administration.
This suggestion, too, was rejected. Testing the same full-year comprehensive content domain produced a predictable, flattering score rise. With each DC-BAS test administration, students recognized more of the content, because they had just been exposed to more of it, so average scores predictably rose. With test scores always rising, it looked like student achievement improved steadily each year. Achieving this contrived score increase required testing students on some material to which they had not yet been exposed, both a violation of professional testing standards and a poor method for instilling student confidence. (Of course, it was also less expensive to administer essentially the same test four times a year than to develop four genuinely different tests.)
5.Synchronize the sequencing of curricular content across the District. DCPS management rhetoric circa 2010 attributed classroom-level benefits to the testing program. Teachers would know more about their students’ levels and needs and could also learn from each other. Yet, the only student test results teachers received at the beginning of each school year was half-a-year old, and most of the information they received over the course of four DC-BAS test administrations was based on not-yet-taught content.
As for cross-district teacher cooperation, unfortunately there was no cross-District coordination of common curricular sequences. Each teacher paced their subject matter however they wished and varied topical emphases according to their own personal preference.
It took DCPS’s Chief Academic Officer, Carey Wright, and her chief of staff, Dan Gordon, less than a minute to reject the suggestion to standardize topical sequencing across schools so that teachers could consult with one another in real time. Tallying up the votes: several hundred school-level District educators favored the proposal, two of Rhee’s trusted lieutenants opposed it. It lost.
6.Offer and require a keyboarding course in the early grades. DCPS was planning to convert all its testing from paper-and-pencil mode to computer delivery within a few years. Yet, keyboarding courses were rare in the early grades. Obviously, without systemwide keyboarding training in computer use some students would be at a disadvantage in computer testing.
Suggestion rejected.
In all, I had polled over 500 DCPS school staff. Not only were all of their suggestions reasonable, some were essential in order to comply with professional assessment standards and ethics.
Nonetheless, back at DCPS’ Central Office, each suggestion was rejected without, to my observation, any serious consideration. The rejecters included Chancellor Rhee, the head of the office of Data and Accountability—the self-titled “Data Lady,” Erin McGoldrick—and the head of the curriculum and instruction division, Carey Wright, and her chief deputy, Dan Gordon.
Four central office staff outvoted several-hundred school staff (and my recommendations as assessment director). In each case, the changes recommended would have meant some additional work on their parts, but in return for substantial improvements in the testing program. Their rhetoric was all about helping teachers and students; but the facts were that the testing program wasn’t structured to help them.
What was the purpose of my several weeks of school visits and staff polling? To solicit “buy in” from school level staff, not feedback.
Ultimately, the new testing program proposal would incorporate all the new features requested by senior Central Office staff, no matter how burdensome, and not a single feature requested by several hundred supportive school-level staff, no matter how helpful. Like many others, I had hoped that the education reform intention of the Rhee-Henderson years was genuine. DCPS could certainly have benefitted from some genuine reform.
Alas, much of the activity labelled “reform” was just for show, and for padding resumes. Numerous central office managers would later work for the Bill and Melinda Gates Foundation. Numerous others would work for entities supported by the Gates or aligned foundations, or in jurisdictions such as Louisiana, where ed reformers held political power. Most would be well paid.
Their genuine accomplishments, or lack thereof, while at DCPS seemed to matter little. What mattered was the appearance of accomplishment and, above all, loyalty to the group. That loyalty required going along to get along: complicity in maintaining the façade of success while withholding any public criticism of or disagreement with other in-group members.
Unfortunately, in the United States what is commonly showcased as education reform is neither a civic enterprise nor a popular movement. Neither parents, the public, nor school-level educators have any direct influence. Rather, at the national level, US education reform is an elite, private club—a small group of tightly-connected politicos and academics—a mutual admiration society dedicated to the career advancement, political influence, and financial benefit of its members, supported by a gaggle of wealthy foundations (e.g., Gates, Walton, Broad, Wallace, Hewlett, Smith-Richardson).
For over a decade, The Ed Reform Club exploited DC for its own benefit. Local elite formed the DC Public Education Fund (DCPEF) to sponsor education projects, such as IMPACT, which they deemed worthy. In the negotiations between the Washington Teachers’ Union and DCPS concluded in 2010, DCPEF arranged a 3 year grant of $64.5M from the Arnold, Broad, Robertson and Walton Foundations to fund a 5-year retroactive teacher pay raise in return for contract language allowing teacher excessing tied to IMPACT, which Rhee promised would lead to annual student test score increases by 2012. Projected goals were not met; foundation support continued nonetheless.
Michelle Johnson (nee Rhee) now chairs the board of a charter school chain in California and occasionally collects $30,000+ in speaker fees but, otherwise, seems to have deliberately withdrawn from the limelight. Despite contributing her own additional scandalsafter she assumed the DCPS Chancellorship, Kaya Henderson ascended to great fame and glory with a “distinguished professorship” at Georgetown; honorary degrees from Georgetown and Catholic Universities; gigs with the Chan Zuckerberg Initiative, Broad Leadership Academy, and Teach for All; and board memberships with The Aspen Institute, The College Board, Robin Hood NYC, and Teach For America. Carey Wright is now state superintendent in Mississippi. Dan Gordon runs a 30-person consulting firm, Education Counsel that strategically partners with major players in US education policy. The manager of the IMPACT teacher evaluation program, Jason Kamras, now works as Superintendent of the Richmond, VA public schools.
Arguably the person most directly responsible for the recurring assessment system fiascos of the Rhee-Henderson years, then Chief of Data and Accountability Erin McGoldrick, now specializes in “data innovation” as partner and chief operating officer at an education management consulting firm. Her firm, Kitamba, strategically partners with its own panoply of major players in US education policy. Its list of recent clients includes the DC Public Charter School Board and DCPS.
If the ambitious DC central office folk who gaudily declared themselves leading education reformers were not really, who were the genuine education reformers during the Rhee-Henderson decade of massive upheaval and per-student expenditures three times those in the state of Utah? They were the school principals and staff whose practical suggestions were ignored by central office glitterati. They were whistleblowers like history teacher Erich Martel who had documented DCPS’ student records’ manipulation and phony graduation rates years before the Washington Post’s celebrated investigation of Ballou High School, and was demoted and then “excessed” by Henderson. Or, school principal Adell Cothorne, who spilled the beans on test answer sheet “erasure parties” at Noyes Education Campus and lost her job under Rhee.
Real reformers with “skin in the game” can’t play it safe.
The author appreciates the helpful comments of Mary Levy and Erich Martel in researching this article.
If there is anything that the recent PISA results show, it’s that the promises by David Coleman, Bill Gates, Michelle Rhee, Betsy Devos, Arne Duncan, Barack Obama, and others of tremendous achievement increases and closing socioeconomic gaps with their ‘reforms’ were completely unfilled. I am copying and pasting here how American students have done on the PISA, a test given in many, many countries, since 2006. There have been tiny changes over the past dozen years in the scores of American students in reading, math, and science, but virtually none have been statistically significant, according to the statisticians who compiled and published the data.
Then again, nearly any classroom teacher you talked to over the past decade or two of educational ‘reforms’ in American classrooms could have told you why and how it was bound to fail.
Valerie Jablow points out that there is an enormous problem with DC public and charter teachers being so harassed that they quit: around 70% of them quit by their 5th year of employment. (She adds that this is probably not a bug, but a feature of the DC teacher evaluation program.) I am reprinting her entire column, but you should subscribe to it yourself.
This Wednesday evening, October 23, at 5:30 pm, the DC state board of education (SBOE), DC’s only elected body with a direct (if relatively powerless) voice on our schools, will take public testimony on teacher retention in DC’s publicly funded schools. (See more information here.)
While public voice is sorely needed in every conversation about our public schools, in this case it’s a bit akin to choosing wallpaper for a burning building.
But that’s hardly SBOE’s fault.
In the wake of years of testimony about horrific treatment of DC teachers, SBOE last year commissioned a study by DC schools expert Mary Levy, which showed terrible attrition of teachers at our publicly funded schools, dwarfing attrition rates nationally.
An update to that 2018 study was just made available by SBOE and will be discussed at the meeting this week.
The update shows that while DCPS teacher and principal attrition rates have dropped slightly recently, they remain very high, with 70% of teachers leaving entirely by the 5-year mark (p. 32). Retention rates for DC’s charter schools are similar to those at DCPS–with the caveat that not only are they self-reported, but they are also not as complete and likely contain errors.
Perhaps the most stunning data point is that more than half of DCPS teachers leaving after 6 years are highly rated (p. 24). This suggests that the exodus of teachers from DC’s publicly funded schools is not merely a matter of weeding out poor performers (as DCPS’s response after p. 70 of this report suggests). Rather, it gives data credence to the terrifying possibility that good teachers are being relentlessly harassed until they give up and leave.
Sadly, that conclusion is the only one that makes sense to me, given that most of my kids’ teachers in my 14 years as a DCPS parent have left their schools–with only a few retiring after many years of service. Most of my kids’ teachers were both competent and caring. Perhaps not coincidentally, they almost always also lacked basic supplies that they ended up buying with their own money; were pressured to teach to tests that would be the basis of their and their principals’ evaluations; and feared reprisal for saying any of that.
(I’m hardly alone in that observation–read some teacher testimony for the SBOE meeting here, including that of a special education teacher, who notes that overwork with caseloads; lack of supplies; and increased class sizes for kids with disabilities are recurring factors at her school that directly lead to teacher burnout.)
In other words, high teacher attrition in DC’s publicly funded schools isn’t a bug but a feature.
Now the real question is why is SBOE apparently the only school leadership body undertaking this work in this manner?
To be fair, DC’s office of the state superintendent of education (OSSE) recently commissioned a report, which showed even higher rates of attrition in DC’s publicly funded schools.
Yet, despite a situation that resembles a full-blown crisis of longstanding proportions, OSSE’s report was weirdly anodyne.
For instance, only 50 of 68 LEAs participated and then, even after citing horrific retention rates, OSSE’s report noted (boldface mine) that “some evidence suggests that DC teacher retention rates may be slightly lower than other cities across the country.”
The report went on to note that “a study of 16 large urban districts found that 81 percent of teachers remained at their schools after one year, compared to 70 percent in DC. National figures suggest that about 84 percent of public school teachers remained at the same school between 2011-12 and the 2012-13 school year.”
Gotta ask:
Is anyone at OSSE at all given pause by the fact that their own citation shows that DC’s teachers are leaving at annual rates more than 10% higher than in comparable urban areas? Or that DC’s 70% annual retention figure above means that a third of DC’s teachers are leaving every year?
Or how about the fact that OSSE’s collaborator on this study, TNTP (founded by former DCPS chancellor Michelle Rhee), has long been the beneficiary of DCPS contracts on teacher performance and training–as well as one of the cheerleaders for rating schools and teachers with test scores, while a former staffer for TNTP recently co-authored a report on DC teacher retention that happily concluded that high teacher turnover can actually increase test scores?
(Yeah–but only for students with teachers receiving the lowest ratings. Yay for us! Oh, and no worries about those kids with those low-rated teachers! Despite the fact that both recent OSSE and SBOE retention reports show that at risk kids in DC are much more likely to have less effective and less experienced teachers who stay for shorter terms, if churning teachers makes for good test scores, perhaps we shouldn’t worry about the collateral damage of taking away the little stability that these kids might otherwise have in their lives. Outcomes, baby, outcomes!)
In fact, OSSE’s recent report on teacher retention appears to be an outgrowth of its recent collaboration with TNTP, the stated goal of which is to “help LEAs develop effective strategies to attract, develop, and retain great teachers to serve their students through robust analysis of staffing data from across the District.”
Of course, that “robust analysis” is only with “LEAs who opt to participate”–which is a charming way to say that whatever OSSE and TNTP have together done on this subject is all, well, voluntary.
Which is kind of like seeing the burning building that is DC teacher retention and not worrying whether everyone has evacuated because choices!
(Or freedom? Hard sometimes to suss out right-wing talking points.)
Indeed, the charter board’s response to the latest SBOE report echoed this (see response after p. 70), noting that “each school pursues its own approach, including its own human capital strategies. In this context, there is no universal “right” rate of attrition, just as there is no universal rate that is too high or too low. The right attrition rate for each school will depend on that school’s approach, their needs and their situation in any given year.”
Despite such official unconcern with the recurring devastation of human capital in our schools, the SBOE is now undertaking to get the council to legislate standardized reporting for teacher attrition, given that we don’t have any standards.
Think about this for a second:
SBOE is asking the council, another elected body with only indirect oversight of schools, to enact legislation to force OSSE to ensure all schools report teacher attrition and retention in a standardized way because we have an emergency here already and no one is telling OSSE to do this. Come to think of it, given the subject matter and its emergency status, you would THINK all this is already OSSE’s obligation (you know, because of that whole mayoral control thingy).
And yet, right now, there is literally only one person in DC who is doing any fulsome reporting of this emergency–and she doesn’t work for OSSE, despite being twice hired by SBOE to report an emergency situation that city education leaders outside SBOE seem to regard as, well, the price of doing business.
So, to recap:
–Horrific teacher retention in all publicly funded schools in DC;
–No standardized and/or mandated reporting of teacher retention in all DC publicly funded schools;
–Teacher harassment and blame for student and school success;
–No official connection of that to poor teacher retention in DC;
–At risk kids bearing the brunt of teacher mobility, including less experienced and effective teachers;
–DC education leaders begging to differ with all of that; and
–A dis-empowered SBOE trying to get both the council and OSSE to actually fix all of that while the mayor is . . . .
A study conducted in Texas with the cooperation of Teach for America claims to pretend that TFA teachers are more effective than their peers. We’ve all heard this claim before, including from frauds like Michelle Rhee, who made up fables about her mythical and fantastic successes during her three years as a TFA newbie in Baltimore.
However, the facts and tables in the report itself shows exactly the opposite, at least for TFA members who are in their first two years (and for many of them, their only two years) in the classroom.
For example, look at the following tables, which I cut and pasted from the report:
Notice what the data is saying in the first four bar graphs above. Dark blue means that the students of that group of TFA teachers were significantly more likely to pass the STAAR test than the students of other, matched, non-TFA teachers. Black means that the students of that group of TFA teachers were NOT more likely to pass, and that this is statistically significant.
Also notice that they do NOT ask the question of whether students of TFA teachers do significantly worse on that test than do students of other teachers. We can only guess.
(1) During the first year that a TFA ‘corps member’ is in the classroom, in 44% to 46% of the cases, their students do NOT do significantly better than their peers on a state-wide standardized test than do the students of non-TFA teachers. We don’t know for a fact that the students actually did WORSE than those taught by non-TFA teachers, but it is certainly a strong possibility. Only in 1/6 to 1/4 of the cases (16% to 26%), do the students of the TFA first-year teachers do significantly better than the students of other, comparable teachers.
(2) Apparently students of second-year TFA teachers in Texas do even worse than those of first-year TFA teachers, especially in reading, because the dark blue sections of the third and fourth bar graphs are significantly smaller than those for the first and second bar graphs, and the black section of bar graph 3 (reading) is much larger than it was in bar graph 1.
Bar graphs 3 and 4 include both first year and second year TFA teachers; since the combined figures for both years 1 and 2 are much worse than for just year 1, that must mean that TFA teachers get worse at preparing their students for the STAAR test during their second year.
(3) However, the relatively few TFA corps members who successfully exit TFA and go on to remain as classroom teachers apparently do much better than the peers that were selected by this study (that’s bar graphs 5 and 6; note the dark blue sections are much larger).
However, let me quote the conclusions written by the authors of this report concerning this graph:
“Students of TFA alumni were significantly more likely to pass STAAR Reading and Math in 77% and 82% of all Reading & Math analyses, respectively. TFA corps members are more effective in Math than Reading. ”
I will let you, the reader, make your own decision as to whether preparing students to pass tests like the PARCC or the STAAR is a worthy goal. But it’s almost all the data that we have.
Those of us with kids in Chevy Chase – DC, either now, in the future, or in the past, have seen many changes in education here in DC, especially since 2007, when the elected board of education was stripped of all powers under PERAA and Chancellor Rhee was appointed by Mayor Fenty.
[I personally went to Junior High School here at Deal back in the early 1960s, taught math in DCPS from 1978 to 2009, including 15 years at Deal (much to my surprise) and my own kids went K-12 in DCPS, graduating from Walls and Banneker, respectively]
Was mayoral control of schools in DC a success? Is the hype we have all heard about rising test scores for real?
We now have statistics from NAEP* for about two decades, and we can compare scores for various subgroups before and after that 2007 milestone.
Did Black students make faster improvements after PERAA than beforehand? Nope. To contrary: their scores were inching up faster *before* 2007 than they have been doing since that time.
Did Hispanic students make faster improvements under the reformers? Nope, again.
How about students whose parent(s) didn’t graduate high school, and/or those who finished grade 12 but either never went to college or else didn’t earn a degree – surely they did better after Rhee, Henderson et al. took over? Again, no.
Then what group of students in Washington DC *did* make more progress on the NAEP after the Reformers took over?
You guessed it, I bet:
White students, and students with parents who earned a college degree.
There has been a dramatic, even shocking, compositional change since 2000 in births across the city, entering cohorts of students, and exit rates from DC schools and the city.
Most noticeably in NW, better educated parents are substantially more likely to have kids in DC, enroll them in DC public schools, and stay past 3rd grade.
Any analysis of test score change needs to grapple with that compositional change.
But more importantly, the compositional change itself is a policy outcome of note, which the DC Council and Mayor have an interest in promoting.
The only evidence one should accept must *at minimum* use longitudinal data on students to compute *learning* as opposed to static achievement, e.g. this analysis of 2008 school closures:
A lot of other things happened 1996-2008 of course, including a rapid expansion of charters, a shrinking proportion of DC residents attending private schools, etc.In 2008 alone, a lot of Catholic schools closed, and some converted to public charter schools.
During this time, we also had a voucher program that produced some gains early on, and then began to lower test scores relative to public options:
All of this is not to say DCPS and charter schools shouldn’t serve less advantaged students better than they do–obviously they should! But the evidence is nuanced, and DC has made huge gains across the board since the 1990’s that make attributing any changes to policy rather than shifting population composition problematic at best.
Interestingly, the NAEP data explorer [https://www. nationsreportcard.gov/ndecore/ xplore/nde]does not report scores for white 8th graders in 1990, 1992, and 1996, presumably because too few were tested. I.e. the means by race show a lot of “‡ Reporting standards not met.“
[I personally attended DCPS (Hyde, Hardy, and School Without Walls) 1976-1989, have 2 children currently in Deal and SWW.]
My previous reply got lost somewhere in cyberspace.
If looking at long-term trends in the NAEP and TUDA is ‘misnaepery’ or ‘jacknaepery’, as Mr Austin would have us believe, then the entire NAEP bureaucracy has been doing just that. (In fact, an entire branch of the National Center for Education Statistics is devoted to, yes, Long Term Trends: https://nces.ed.gov/nationsreportcard/ltt/ )
It’s a laughable idea that we could just use the tests chosen by DCPS and later by OSSE and administered every year, to tell how good DC public or charter schools are, over time. First of all, the tests administered here have changed dramatically. Back in the 1990s it was the CTBS. Then it was the SAT-9, developed by a different company. Then it was the DC-CAS, again, a different vendor. Now we have the PARCC produced by yet another vendor. We also know that in the past there has been major fraud with these tests, committed by adults, in order to gain bonuses and keep their jobs. We also have no way of comparing DC with any other city or state using those tests, since only a handful of states even use the PARCC and for all I know, their cut scores and questions might be different from what we use here in DC.
The idea of measuring median student improvement from year to year might appear to have some merit, until you talk to students and teachers involved. You discover that many of the older students see no reason to take the tests seriously; they bubble in, or click on, answers as fast as possible, without reading the questions, in order to be free to leave the room and go do something else. Any results from that test are simply unreliable, and it is simply not possible to tell whether DC education policies have improved over time based on the PARCC, DC-CAS, SAT-9, or CTBS, no matter what sort of fancy statistical procedures are employed.
With the NAEP, on the other hand, there has never been any suggestion of impropriety, and the same agency has been devising, administering, and scoring these tests for decades. We have no other nation-wide test that has been systematically given to a random sample of students for any length of time.
Obviously the 4th or 8th graders who took the NAEP in 2017 were not the same ones who took it in 2015. (Duh!) However, we do in fact have a record of NAEP scores in every state and DC since the 1990s, and they are also broken down by lots of subgroups. Obviously DC is gentrifying rapidly, and there are more white students in DCPS than there were 10 or 20 years ago. If we trace the various subgroups (say, African-American students, or Hispanics, or students whose parents didn’t finish high school, or whatever group you like), you can watch the trends over time in each subgroup. However, Mr Austin does inadvertently raise one valid point: since the proportion of black students in DC is decreasing, and the proportion of white students with college-educated parents is rising, then the natural conclusion would be that this gentrification has *inflated* overall scores for 4th and 8th grade students in DC (and DCPS), especially since 2007. Which is more evidence that ‘reform’ is not working. Not evidence that we should throw the scores out and ignore them completely.
Those trends show something quite different from what Mayor Bowser keeps proclaiming. For one thing, if you look at the simple graphs that I made (and you can examine the numbers yourselves) you can see that any improvements overall in DC, or for any subgroups, began a decade before the ‘reformers’ took over DC schools. ( see https://bit.ly/2K3UyZ1 to begin poking around.) Secondly, for most of the subgroups, those improvements over time were greater before Rhee was anointed Chancellor. Only two groups had better rates of change AFTER Rhee: white students, and those with parents with college degrees – the ones that are inflating overall scores for DC and DCPS during the last decade.
I would note also that the previous writer’s salary is paid by one of the Reform organizations supported by billionaires Gates and Arnold. You can look at the funding page yourself ( page 3 at https://urbn.is/2II1YQQ ). I suspect that when ‘reform’ advocates say not to look at our one consistent source of educational data, it’s because they don’t like what the data is saying.
GFBrandenburg's Blog 