Weekly Roundup of Educational Resistance by Bob Schaeffer

{As usual, this list is collected and distributed by Bob Schaeffer, not by me.}

The U.S. Senate has joined the House of Representatives in responding to growing, grassroots pressure by voting to overhaul “No Child Left Behind” (NCLB). The bills passed by both the Senate and House reflect widespread rejection of failed top-down, test-and-punish strategies as well as the “NCLB on steroids” waiver regime dictated by Arne Duncan. While neither version is close to perfect from an assessment reform perspective, each makes significant progress by rolling back federally mandated high-stakes, eliminating requirements to evaluate educators based on student test scores, and recognizing opt-out rights. FairTest and its allies will closely monitor the conference committee working on compromise language to make sure the gains remain in the final bill sent to President Obama — the alternative is to keep the yoke of NCLB-and-waivers in place for at least two more years, if not much longer. Meanwhile, organizers in many states are keeping the spotlight on the problems of test overuse and misuse, modeling better practices and winning additional policy victories.

Remember that back issues of these weekly updates are archived at:http://fairtest.org/news/other

National End High-Stakes Testing to Help Fix Public Education: Key Civil Rights Leader

http://thehill.com/blogs/congress-blog/education/247770-fix-public-education-end-high-stakes-testing-pass-esea
National U.S. Senate Rejects Proposal to Give Federal Government More Say in Identifying “Failing” Schools
http://www.washingtonpost.com/local/education/senate-rejects-effort-to-give-feds-more-say-in-identifying-failing-schools/2015/07/15/f6ad9ba2-2a6b-11e5-a5ea-cf74396e59ec_story.html
National Both House and Senate NCLB Overhaul Bills Allow for Penalty-Free Test Opt Out
https://deutsch29.wordpress.com/2015/07/17/both-house-and-senate-esea-bills-allow-for-opt-out-without-penalty/
National “Race to the Top:” Lofty Promises and Top-Down Regulation Brought Few Good Changes to America’s Schools
http://educationnext.org/lofty-promises-little-change-americas-schools/

California
Exit Exam on Way Out
http://www.hidesertstar.com/news/article_ee092084-2cfa-11e5-98d3-fbf7e03679c0.html

Colorado
Two Small Districts Set Opt Out Records
http://co.chalkbeat.org/2015/07/20/two-small-districts-set-the-record-for-opting-out/#.Va1_rZdLUZw

Connecticut
Opposition Coalesces Against Smarter Balanced Tests
http://www.newmilfordspectrum.com/news/article/Opposition-coelesces-regarding-school-testing-6393921.php

Delaware
Governor Vetoes Opt-Out Bill; State PTA Pushed for Override Vote
http://www.delawareonline.com/story/news/education/2015/07/16/markell-vetoes-testing-opt-bill/30243869/

Georgia
More than 10,000 Young People Who Did Not Pass Grad. Test Recently Received Diplomas
http://jacksonville.com/news/georgia/2015-07-17/story/georgia-hands-diplomas-more-10000-people-who-couldnt-pass-high-school

Hawaii Teachers Fight Evaluations Based on Student Test Scores
http://www.civilbeat.com/2015/07/she-fought-hawaiis-new-way-of-evaluating-teachers-and-won/

Illinois
Why Common Core Tests Are Harmful to Students
http://dianeravitch.net/2015/07/20/gerri-k-songer-explains-why-common-core-tests-are-actually-harmful-to-students/

Iowa
Third-Grade Promotion Test Pushes Reading Down Into Kindergarten
http://www.desmoinesregister.com/story/news/education/2015/04/20/early-reading-education-reform/26100783/

Louisiana
Fight to Make Charter School Disclose What Test It Uses for Kindergarten Entry
http://hechingerreport.org/when-a-top-nola-charter-wont-reveal-its-admission-test-for-kindergarten/

Minnesota
Test Cuts Came After Thorough Debate
http://www.startribune.com/editorial-counterpoint-school-testing-cuts-were-fully-debated/316368021/

Missouri
Exam Scores Don’t Tell Full Story of Teacher Preparedness
http://www.examiner.net/article/20150715/OPINION/150719341/-1/sports

Ohio
Time Allocated to New State Tests Cut in Half
http://www.cleveland.com/metro/index.ssf/2015/07/ohios_common_core_math_and_english_tests_will_be_cut_to_3_hours_each.html

Nevada After Testing System Breakdown, State to Hire New Assessment Vendor
http://blogs.edweek.org/edweek/state_edwatch/2015/07/after_testing_problems_nevada_set_to_hire_new_assessment_vendor.html

New Hampshire Schools Can Replace Smarter Balanced Tests with ACT or SAT
http://nhpr.org/post/nh-schools-can-replace-smarter-balanced-test-sat-and-act

New Jersey
Be Wary of New State Teacher Ratings
http://www.app.com/story/opinion/2015/07/17/teacher-ratings/30307937/

New Mexico
Court Rejects Suit Seeking to Strip Pearson’s Common Core Testing Contract
http://blogs.edweek.org/edweek/marketplacek12/2015/07/court_hands_major_victory_to_parcc_pearson_in_challenge_by_vendor.html

New York
High School Models Authentic Assessment
http://www.edweek.org/ew/articles/2015/07/17/nyc-high-school-strives-for-authentic-assessment.html
New York Opt Out Movement Plans to Ratchet Up Actions Against Standardized Exam Overkill
http://blog.timesunion.com/capitol/archives/238624/opt-outers-will-continue-to-protest-tests/
New York Pending NCLB Overhaul Offers Hope to Reduce State’s Testing Obsession
http://www.lohud.com/story/opinion/2015/07/17/hope-reduce-testing-obsession-nclb/30290533/

North Carolina State’s Largest District Cuts Back Local Test Mandates
http://www.newsobserver.com/news/local/education/article27865318.html
North Carolina Cautions About Test-Score-Based Teacher Pay
http://www.newsobserver.com/news/local/community/chapel-hill-news/chn-opinion/article27512254.html

Oregon
Students Can Meet Graduation Requirement with Work Samples in Their Home Language
http://www.oregonlive.com/education/index.ssf/2015/07/oregon_students_can_qualify_fo.html

Pennsylvania
Questions Mount About Using Volatile Test Results to Evaluate Teachers and Schools
http://thenotebook.org/blog/158810/educators-pennsylvania-pssa-test-score-plunge
Pennsylvania Teachers to School Board: Standardized Testing is Harming Students
http://lancasteronline.com/news/local/cocalico-teachers-to-school-board-standardized-testing-is-harming-students/article_dc32651c-2e4b-11e5-803e-0be9cdf1e4bc.html

Rhode Island
What Tests Like PARCC Do Not Measure
http://www.golocalprov.com/news/guest-mindsetter-joel-hellmann-what-parcc-doesnt-measure

Tennessee
Teachers School Governor on Testing and Evaluations
http://tn.chalkbeat.org/2015/07/16/tennessee-teachers-school-haslam-on-testing-evaluations-during-first-teachers-cabinet-meeting/#.VahPb5dLUZw
Tennessee Local School Board to Take Up Opt Out Resolution
http://springhillhomepage.com/wcs-board-delays-talks-on-over-testing-cms-4594

Texas
New Test Leading Fewer to Get GEDs
https://www.texastribune.org/2015/07/14/education-advocates-say-new-computer-based-ged-too/

Washington State Testing Revolt Pushes State Into Uncharted Waters
http://wutc.org/post/testing-revolt-washington-state-brings-feds-uncharted-waters
Washington Over-Testing is a Flawed Strategy
http://www.dailyrecordnews.com/familyandeducation/is-there-too-much-testing/article_15326290-2cdf-11e5-b9a9-637e48088a15.html

“How Many Tests Can a Child Withstand?” — with apologies to Bob Dylan
http://www.washingtonpost.com/blogs/answer-sheet/wp/2015/07/14/with-apologies-to-bob-dylan-a-song-how-many-tests-must-a-child-withstandbefore-we-can-kill-this-scam/

The Beatings in Education Will Continue Until Morale Improves
http://www.huffingtonpost.com/steve-nelson/the-beatings-will-continu_b_7795784.html

Bob Schaeffer, Public Education Director
FairTest: National Center for Fair & Open Testing
office-   (239) 395-6773   fax-  (239) 395-6779
mobile- (239) 699-0468
web-  http://www.fairtest.org

Advertisements

Important Article Shows that ‘Value-Added’ Measurements are Neither Valid nor Reliable

As you probably know, a handful of agricultural researchers and economists have come up with extremely complicated “Value-Added” Measurement (VAM) systems that purport to be able to grade teachers’ output exactly.

These economists (Hanushek, Chetty and a few others) claim that their formulas are magically mathematically able to single out the contribution of every single teacher to the future test scores and total lifetime earnings of their students 5 to 50 years into the future. I’m not kidding.

Of course, those same economists claim that the teacher is the single most important variable affecting their student’s school and trajectories – not family background or income, nor peer pressure, nor even whole-school variables. (Many other studies have shown that the effect of any individual teacher, or all teachers, is pretty small – from 1% to 14% of the entire variation, which corresponds to what I found during my 30 years of teaching … ie, not nearly as much of an impact as I would have liked [or feared], one way or another…)

Diane Ravitch has brought to my attention an important study by Stuart Yen at UMinn that (once again) refutes those claims, which are being used right now in state after state and county after county, to randomly fire large numbers of teachers who have tried to devote their lives to helping students.

According to the study, here are a few of the problems with VAM:

1. As I have shown repeatedly using the New York City value-added scores that were printed in the NYTimes and NYPost, teachers’ VAM scores vary tremendously over time. (More on that below; note that if you use VAM scores, 80% of ALL teachers should be fired after their first year of teaching) Plus RAND researchers found much the same thing in North CarolinaAlso see this. And this.

2. Students are not assigned randomly to teachers (I can vouch for that!) or to schools, and there are always a fair number of students for whom no prior or future data is available, because they move to other schools or states, or drop out, or whatever; and those students with missing data are NOT randomly distributed, which pretty makes the whole VAM setup an exercise in futility.

3. The tests themselves often don’t measure what they are purported to measure. (Complaints about the quality of test items are legion…)

Here is an extensive quote from the article. It’s a section that Ravitch didn’t excerpt, so I will, with a few sentences highlighted by me, since it concurs with what I have repeatedly claimed on my blog:

A largely ignored problem is that true teacher performance, contrary to the main assumption underlying current VAM models, varies over time (Goldhaber & Hansen, 2012). These models assume that each teacher exhibits an underlying trend in performance that can be detected given a sufficient amount of data. The question of stability is not a question about whether average teacher performance rises, declines, or remains flat over time.

The issue that concerns critics of VAM is whether individual teacher performance fluctuates over time in a way that invalidates inferences that an individual teacher is “low-” or “high-” performing.

This distinction is crucial because VAM is increasingly being applied such that individual teachers who are identified as low-performing are to be terminated. From the perspective of individual teachers, it is inappropriate and invalid to fire a teacher whose performance is low this year but high the next year, and it is inappropriate to retain a teacher whose performance is high this year but low next year.

Even if average teacher performance remains stable over time, individual teacher performance may fluctuate wildly from year to year.  (my emphasis – gfb)

While previous studies examined the intertemporal stability of value-added teacher rankings over one-year periods and found that reliability is inadequate for high-stakes decisions, researchers tended to assume that this instability was primarily a function of measurement error and sought ways to reduce this error (Aaronson, Barrow, & Sander, 2007; Ballou, 2005; Koedel & Betts, 2007; McCaffrey, Sass, Lockwood, & Mihaly, 2009).

However, this hypothesis was rejected by Goldhaber and Hansen (2012), who investigated the stability of teacher performance in North Carolina using data spanning 10 years and found that much of a teacher’s true performance varies over time due to unobservable factors such as effort, motivation, and class chemistry that are not easily captured through VAM. This invalidates the assumption of stable teacher performance that is embedded in Hanushek’s (2009b) and Gordon et al.’s (2006) VAM-based policy proposals, as well as VAM models specified by McCaffrey et al. (2009) and Staiger and Rockoff (2010) (see Goldhaber & Hansen, 2012, p. 15).

The implication is that standard estimates of impact when using VAM to identify and replace low-performing teachers are significantly inflated (see Goldhaber & Hansen, 2012, p. 31).

As you also probably know, the four main ‘tools’ of the billionaire-led educational DEform movement are:

* firing lots of teachers

* breaking their unions

* closing public schools and turning education over to the private sector

* changing education into tests to prepare for tests that get the kids ready for tests that are preparation for the real tests

They’ve been doing this for almost a decade now under No Child Left Untested and Race to the Trough, and none of these ‘reforms’ have shown to make any actual improvement in the overall education of our youth.


Jay Mathews’ Fawning Column

I had no idea that was coming; in fact, it looked as though the lame-stream media had decided to ignore the entire matter.

Mathews makes entirely too much of my researching abilities. It was Ed Harris (thanks!) who alerted me to that report. My understanding of statistics is of an entirely elementary nature.

Unlike some folks (eg MR or leaders of certain religious groups) I make no claim to making miracles happen, to omniscience, or to inerrancy. I like to think I had some successes in the classroom, and I know for sure that I had some failures. Some kids and their parents liked my approach, and some hated my guts. I tried (with some success, occasionally) to show how math was useful in real life. Like most teachers, I worked hard, but always found that the amount of work required was at least double the amount of time I had available and could possibly provide.

Whenever I tried to do any statistical stuff with my own students’ accomplishments on final exams, on standardized tests, or even letter grades, after a year with me,  I was consistently mystified: nothing ever seemed to correlate with anything, or if they did, the correlation coefficients were extremely low. My personal experience in this regard leads me to suspect that there is so much unexplained, seemingly random, variation in human performance, desires, and so on that any sort of ‘value-added’ measurement is going to be bogus.
For the record, since Rhee and her colleague team-taught the entire 3rd grade class at Harlem Park during her last year, then, if you believe the rhetoric of Hanushek, Rhee, and others, then she and her co-teacher were responsible for the growth (or not) of the entire cohort. (BTW: who was that other ‘miracle’-worker? Evidently someone a lot less arrogant and prone to self-promotion than Rhee!)
To quote one of my posts:

“The cohort that started the first grade at Harlem Park in 1992-1993 had 84 students, probably 3 or 4 distinct classes.

“When they arrived in the second grade in 1993-1994 and endured Michelle Rhee’s second failed year of teaching, they still had 83 students – probably 3 or 4 classes again.

“But when this cohort arrived in the third grade in 1994-1995, Rhee’s “miracle year”, their numbers dropped by nearly half, to only 44 students. I doubt strongly that so many students dropped dead. I can’t prove it, but I would not be surprised if the school (and Rhee) ‘counseled out’ the ones who were doing poorly, and kept the ones who had high test scores.”

So, even though half of the students ‘disappeared’, the most that miracle-worker Rhee could do is to get the rest up to somewhere near the 50th percentile.

Analysis of ‘Value-Added’ Measurements

Good article:

http://blog.ednewscolorado.org/2010/11/29/proceed-with-caution-on-value-added/

Published in: on December 18, 2010 at 4:08 pm  Leave a Comment  
Tags: , , ,

Impossible Goals for Schools

From “Grading Education: Getting Accountability Right” by Richard Rothstein, Rebecca Jacobsen, and Tamara Wilder, (Teachers College Press, 2008) pages 70-71

Impossible Goals for Schools

Inadequate schools are only one reason disadvantaged children perform poorly. They come to school under stress from high-crime neighborhoods and economically insecure households. Their low-cost day care tends to park them before televisions, rather than provide opportunities for developmentally appropriate play. They switch schools more often because of inadequate housing and rents rising faster than parents’ wages. They have greater health problems, some (like lead poisoning or iron-deficiency anemia) directly depressing cognitive ability, and some (like asthma and vision difficulties) causing more absenteeism or inattentiveness. Their household include fewer college-educated adults to provide more sophisticated intellectual environments, and their parents are less likely to expect academic success.* Nearly 15% of the black-white score gap can be traced to differences in housing mobility, and 25% to differences in child and maternal health.**

Yet contemporary test-based accountability policies that establish the goal of all students being proficient require that school improvement alone – higher expectations, better teachers, improved curriculum, and more testing – should raise all children to high levels of achievement, poised for college and professional success. Natural human variability would still distinguish children, but these distinctions would have nothing to do with family disadvantage. If true, there really would he no reason for progressive housing or health and economic policies. The nation’s social and economic problems would take care of themselves, by the next generation.

Teachers of children who come to school hungry, scared, abused, or ill consider this absurd. But increasingly, in our test-based accountability environment, pronouncements of politicians and some educational leaders intimidate teachers from acknowledging the obvious. Instead, teachers are expected to repeat the mantra “all children can learn,” a truth carrying the false implication that the level to which children learn has nothing to do with their starting points. Policy makers and school administrators warn teachers that any mention of children’s socioeconomic disadvantages only “makes excuses” for teachers’ own poor performance.

Of course, there are better and worse schools and better and worse teachers. And of course, some disadvantaged children excel more than others. But our federal and state test-based accountability policies, anchored to the demand for a single standard of proficiency for all students, regardless of background, have turned these obvious truths into the fantasy that teachers can wipe out socioeconomic differences among children simply by trying harder.

Denouncing schools as the chief cause of American inequality – in academic achievement, thus in the labor market, and thus in life generally – stimulates cynicism among teachers who are expected to act on a theory they know to be false. Many dedicated and talented teachers are abandoning education; they may have achieved exceptional results with disadvantaged children but, with state and federal proficiency bars set so impossibly high, even these teachers are labeled failures.

Continuation of the rhetoric of test-based accountability will also erode support for public education. Under pressure, educators now publicly vow they can eliminate achievement gaps, but they will inevitably fall short. When these educators then fail to fulfill the impossible expectations they themselves have endorsed, the reasonable conclusion can only be that they and their colleagues in public education are hopelessly incompetent.

* Discussed at length in Richard Rothstein, Class and Schools: Using Social, Economic, and Educational Reform to Close the Black-White Achievement Gap (Teachers College Press, 2004); see also Susan B. Neuman, Changing the Odds for Children at Risk: Seven Essential Principles of Education Programs that Break the Cycle of Poverty (Praeger, 2008); and Susan B Neuman, “Education Should Lift All Children” (Detroit Free Press, July 31, 2008).
** For an estimate of the effect of mobility, see Eric Hanushek, John Kain, and Steven Rivkin, “Disruption versus Tiebout*** Improvement: The Costs and Benefits of Switching Schools” (Journal of Public Education, 2004, 88(9-10): 1721-46). For an estimate of the effect of child and maternal health, see Janet Currie, “Health Disparities and Gaps in School Readiness” (The Future of Children, 2005m 15(1): 117-38)
*** A technical economic model that I [GFB] have never heard of before. Here is the beginning of the Wikipedia entry: The Tiebout model, also known as Tiebout sorting, Tiebout migration, or Tiebout hypothesis, is a positive political theory model first described by economist Charles Tiebout in his article “A Pure Theory of Local Expenditures” (1956). The essence of the model is that there is in fact a non-political solution to the free rider problem in local governance.

Results from an actual randomized study at the USAFA

A very interesting study done at the US Air Force Academy and published here, contradicts most of what Hanushek, Kamras, Rhee et al have said about ‘value-added’ measures of teaching.

Some of its conclusions:

“…our results indicate that professors who excel at promoting contemporaneous student achievement [that is, who do well at what Rhee and Kamras would call ‘value-added scores’], on average, harm the subsequent performance of their students in more advanced classes.

“Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value-added but positively correlated with follow-on course value-added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow-on related curriculum.

“Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses). Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.

“Similar to elementary and secondaryschool teachers, who often have advance knowledge of assessmentcontent in high-stakes testing systems, all professors teaching a given course at USAFA have an advance copy of the exam before it is given. Hence, educators in both settings must choose how much time to allocate to tasks that have great value for raising current scores but may have little value for lasting knowledge.

“Using our various measures of quality to rank-order professors leads to profoundly different results.”

“the correlation between introductory calculus professor value-added in the introductory and follow-on courses is negative, r=-0.68. Students appear to reward contemporaneous course value-added, r=+0.36, but punish deep learning, r=-0.31.”

(In other words, one method of measuring professor quality will rank them in one way, but if you use a different method of measuring professor quality, you get a ranking that is profoundly different.)

In this study, students and instructors were randomly assigned to all of the courses, and students had to take follow-up courses in, say, calculus and chemistry, regardless of how much they liked the subject or not, or how well they did in their previous course. Thus, once they were at the Air Force Academy, there was no self-selection by professors or by students. This eliminated an element that probably confounds prior studies.

Source:

http://www.journals.uchicago.edu/doi/pdf/10.1086/653808

%d bloggers like this: