A closer look at charter and regular public school enrollments, percentages of students at risk, and percentages of students ‘proficient’

Here is another look at the brand-new data concerning four variables in the District of Columbia schools, about which I wrote a couple of days ago. The difference here is that the dots representing the schools are more-or=less proportional to the size of the student body.

1. Is this a regular public school, or a charter school (blue or red):

2. What fraction of the kids at that school are officially considered to be At Risk? (That’s the scale along the x-axis at the bottom of the page)

3. What is the average percentage of the kids at that school are ‘proficient’ in reading and math on the DC-CAS? (That’s the scale along the y-axis at the left-hand side of the page)

4. How big is the school? (That’s the size of the dot, more or less; the legend is at the bottom left-hand corner of the graph)

Time spent looking carefully at this graph will be well-spent. If you click on it, it will expand.

It will certainly show that charter schools have not revolutionized education for the better in DC: for both types of schools, there remains a very strong, negative correlation between the percentages of kids At Risk and ‘pass’ rates on the DC-CAS.

Note that most schools have between 200 and 500 students and that most of the ones that are smaller are actually charter schools. As I wrote a couple of days ago, the schools with the largest fraction of At-Risk students (say, over 2/3 of the student body) are almost all regular DC public schools.

On the second graph, which is otherwise identical to the first, I’ve labeled some of the larger schools.

fixed bicolor, size of school and at risk vs average dc cas 2014 proficiency, both regular public and charter, dc

Here is the one with names of some of the larger schools, so you can see how individual schools fall on this graph.

(Sorry, I there was not enough room to label every single one, and my non-existent HTML skills won’t allow me to make it so that any of the dots are clickable. If any of my readers know how to do that and would like to offer to make that happen, then please let me know in the comments.)

again fixed and revised names and bicolor, size of school and at risk vs average dc cas 2014 proficiency, both regular public and charter, dc

And here is the entire data table. So you can see where every single school lies on these three dimensions.

(PS: I added a few more names of schools and corrected four other small errors, two pointed out by an alert reader.. 2/22/2015)

How Well are Charter Schools in DC Educating Students Who are Officially At-Risk?

The results may surprise you.

To answer this question, I used some recent data. I just found out that the DC City Council has begun requiring that schools enumerate the number of students who are officially At-Risk. They define this as students who are

“homeless, in the District’s foster care system, qualify for Temporary Assistance for Needy Families (TANF) or the Supplemental Nutrition Assistance Program (SNAP), or high school students that are one year older, or more, than the expected age for the grade in which the students are enrolled.” (That last group is high school students who have been held back at least one time at some point in their school career.)

So, it’s a simple (but tedious) affair for me to plot the percentage of such at risk students, at each of the roughly 200 publicly-funded schools in Washington, DC, versus the average percentage of students who were proficient or advanced in math and reading on the 2014 DC-CAS.

I was rather shocked by the results. Here are my main conclusions:

1. For almost all of the schools, to get a rough idea of the percent of students passing the DC-CAS, simply subtract 90% minus the number of students ‘At-Risk’. The correlation is very, very strong.

2. There are only THREE DC charter schools with 70% or more of their students At-Risk, whereas there are THIRTY-ONE such regular public schools. So much for the idea that the charter schools would do a better job of educating the hardest-to-reach students (the homeless, those on food stamps, those who have already failed one or more grades, etc).

3. The only schools that have more than 90% of their students ‘passing’ the DC-CAS standardized tests remain, to this day, the small handful of schools in relatively-affluent upper Northwest DC with relatively high percentages of white and Asian students..(Unless you include Sharpe Health school, where students who cannot feed or dress themselves or hold a pencil are somehow deemed ‘proficient’ or ‘advanced’ by methods I can only guess at…)

4. As I’ve indicated before, it appears that for the most part, DC’s charter schools are mostly enrolling smaller percentages of At-Risk, high-poverty students but higher fractions of the students in the middle of the wealth/family-cohesion spectrum than the regular DC public schools. There are a few exceptions among the charter schools: BASIS, Yu Ying, Washington Latin and a few others are succeeding in attracting families and students at the high end of the socio-economic and academic scales.

5. It looks like we are now turning into a tripartite school system: one for affluent and well-educated familes (relatively high fractions of whites and Asians; mostly but not all in regular Ward 3 public schools); one for those in the middle (mostly blacks and hispanics, many enrolled in charter schools), and one for those at the seriously low end of the socio-economic spectrum, overwhelmingly African-American, largely At Risk, and mostly in highly-segregated regular public schools.

Very, very sad.

Here is the graph that sums it all up. Click on it to see a larger version.

bicolor, at risk vs average dc cas 2014 proficiency, both regular public and charter, dc

In blue we have the regular public schools of Washington DC for which I have DC-CAS data for 2014, from grades 3 through 8 and grade 10. In red we have the privately-run but publicly-funded charter schools. Along the horizontal axis, we have the percentage of students who are officially At Risk as defined by the DC CIty Council. Along the vertical axis, we have the average percentage of students who scored ‘proficient’ or ‘advanced’ in math and reading on the DC-CAS at those schools. The green line is the line of best fit as calculated by Excel. Notice that the data points pretty much follow that green line, slanting down and to the right.

To nobody’s surprise, at both the charter and regular public schools, on the whole, the greater the percentage of students at a school who are At Risk, the smaller the percentage of students who ‘pass’ the DC-CAS standardized tests.

The colors do help us see that at the far right-hand end of the graph, there are lots of blue dots and only a small number of red ones. This means that the vast majority of schools with high percentages of At Risk students are regular DC public schools. You could interpret that to mean that parents in more stable families in those neighborhoods are fleeing from what they see as the bad influence of potential classmates who are extremely poor, homeless, have already repeated a grade, and so on, and are flocking to charter schools who have the freedom to expel or ‘counsel out’ such students and to impose a relatively strict behavior code that the DC Council forbids the regular public schools from using. (Their latest initiative is to forbit ALL out-of-school suspensions, no matter what…)

Dots that are above the slanted green line supposedly represent schools that are doing a better job at teaching to the tests than would be predicted by the At-Risk status alone. Dots below the line are doing a worse job than would be predicted. Notice that there are dots of both colors both above and below the line.

=====

I wish to thank the indefatigable Mary Levy for collecting and passing on this data. You can find the original data source at the OSSE website, but I’ve saved the larger table (all 2008-2014 DC-CAS data) on Google Drive at this link. I took the average of the percentage of students ‘passing’ the DC-CAS in math and in reading as the proficiency rate. The note on the at-risk data table reads as follows:

Data Source: SY2013-14 student-level data from OSSE. The list includes DCPS traditional, DCPS citywide specialized, DCPS selective schools, and public charter schools, but excludes any DCPS or public charter adult education or alternative school. The definition of at risk students includes students who are homeless, in the District’s foster care system, qualify for Temporary Assistance for Needy Families (TANF) or the Supplemental Nutrition Assistance Program (SNAP), or high school students that are one year older, or more, than the expected age for the grade in which the students are enrolled.

Listing of Educational Bloggers

This is a list of the blogs maintained at the present time by some fellow-activist teachers and others.

Enjoy!

BLOGGER NAME BLOG NAME BLOG WEBSITE
A Teacher on Teaching A Teacher on Teaching http://ateacheronteaching.blogspot.com/
Aaron Barlow Aaron Barlow http://academeblog.org/author/aaronbarlow/ or http://audsandens.blogspot.com/
Accountable Talk Accountable Talk http://www.accountabletalk.com/
Adam Bessie Automated Teaching Machine http://adambessie.com/
Alan Singer Alan Singer http://www.huffingtonpost.com/alan-singer/
Alexandra Miletta Alexandra Miletta http://alexandramiletta.blogspot.com
Alice Mercer Reflections on Teaching http://mizmercer.edublogs.org
Allan Jones Allan Jones https://www.facebook.com/groups/1398276720427252/
Amy Moore Amy Moore http://www.desmoinesregister.com/topic/065294af-047d-4b86-beb4-0d401eb82096/
Andy Spears Tennessee Education Report http://tnedreport.com/
Ani McHugh Teacherbiz http://teacherbiz.wordpress.com
Ann Policelli Cronin Ann Policelli Cronin http://reallearningct.com/
Anne Tenaglia Teacher’s Lessons Learned http://teacherslessonslearned.blogspot.com/
Anthony Cody Anthony Cody http://www.livingindialogue.com/
Arthur Getzel The Public Educator (aka liberalteacher) http://thepubliceducator.com/
Arthur Goldstein NYCEducator http://nyceducator.com/
Arthur H. Camins Arthur H. Camins http://www.arthurcamins.com/
Audrey Amrein-Beardsley VAMboozled http://vamboozled.com/
Aurelio M. Montemayor Parent Leadership in Education http://parentleadershipined.blogspot.com/
Badass Teachers Association (Marla Kilfoyle, Melissa Tomlinson) Badass Teachers Association http://badassteachers.blogspot.com/ and http://www.badassteacher.org/
Barbara Madeloni Educators for a Democratic Union http://www.educatorsforademocraticunion.com/
Barbara McClanahan readingdoc http://readingdoc.wordpress.com/
Betsy Combier Parent Advocatees http://www.parentadvocates.org/
Big Education Ape Big Education Ape http://bigeducationape.blogspot.com/
Bill Betzen School Achieve Project http://schoolarchiveproject.blogspot.com/
Bill Boyle Educarenow http://educarenow.wordpress.com/
Bob Sikes Scathing Purple Musings http://bobsidlethoughtsandmusings.wordpress.com/
Bob Valiant Defend-Ed http://defend-ed.org/
Bonnie Cunard Continuing Change http://gatorbonbc.wordpress.com/ orhttp://bonniecunardmargolin.weebly.com/
Bonny Buffington BBBloviations http://www.bbbloviations.blogspot.com/
Brett Bymaster Stop Rocketship http://www.stoprocketship.com
Brett Dickerson Life At the Intersections http://www.brettdickerson.net/
Brian Cohen Making the grade blog http://www.bncohen.com/
Brian Redmond rsbandman http://rsbandman.wordpress.com
Bruce Baker School Finance 101 http://schoolfinance101.wordpress.com/
Bruce Bowers Reflections on teaching and learning www.tremphil.com
Carol Burris Carol Burris http://roundtheinkwell.com/ and Answer Sheet
Chaz Chaz’s School Daze http://chaz11.blogspot.com/
Chris Cerrone Children should not be a number http://www.nystoptesting.com/
Chris Guerrieri Jaxkidsmatter http://jaxkidsmatter.blogspot.com/
Chris Thinnes Chris Thinnes http://chris.thinnes.me
Christian Goering Edusanity http://www.edusanity.com/
Christopher Martell On Social Studies and Education http://christophermartell.blogspot.com
Christopher Tienken Christopher Tienken http://christienken.com/blog/
Christopher Wooleyhand Common Sense School Leadership http://christopherwooleyhand.edublogs.org
Claudia Swisher Claudia Swisher http://fourthgenerationteacher.blogspot.com/
Cynthia Liu K12NN News Network http://k12newsnetwork.com/
Dan McConnell Truth and Consequences http://dan-mcconnell.blogspot.com/
Daniel Katz Daniel Katz http://danielskatz.net/
Darcie Cimarusti Mother Crusader http://mothercrusader.blogspot.com/
David Chura Kids in the System http://kidsinthesystem.wordpress.com/
David Cohen InterACT:  Accomplished California Teacher http://accomplishedcaliforniateachers.wordpress.com/
David Ellison A Teacher’s Mark’s http://ateachersmarks.blogspot.com/
David Greene DCG MENTORING https://dcgmentor.wordpress.com 
Debbie Forward PFF Faculty Lounge http://pfffacultylounge.wordpress.com/
Deborah McCallum Big Ideas in Education http://bigideasineducation.ca/
Deborah Meier Deborah Meier http://blogs.edweek.org/edweek/Bridging-Differences/
Demian Godon Reconsidering TFA https://reconsideringtfa.wordpress.com/
Derek Black Education Law Prof Blog http://lawprofessors.typepad.com/education_law/
Diane Aoki The Teacher I Want to Be http://dianeaoki.blogspot.com/
Diane Ravitch Diane Ravitch http://dianeravitch.net
DOE Nutes DOE Nuts Blog http://nycdoenuts.blogspot.com/
Don Russell Lifting The Curtain http://liftingthecurtainoneducation.wordpress.com/
Dora Taylor Seattle Education http://seattleducation2010.wordpress.com/
Doug Martin Doug Martin http://www.schoolsmatter.info/ 
Edward Berger Edward Berger http://edwardfberger.com/
Elizabeth Rose Yo Miz http://yomizthebook.com/
Francesco Portelos Educator Fights Back  or Don’t Tread on Educators http://dtoe.org/ or http://protectportelos.org/
Fred Klonsky Fred Klonsky http://preaprez.wordpress.com/
Gary Rubinstein Gary Rubinstein https://garyrubinstein.wordpress.com/
Gene Glass Education in Two Words http://ed2worlds.blogspot.com/
George Schmidt Substance News http://www.substancenews.net/
George Wood George Wood http://www.essentialschools.org/
Gerri Songer Gerri Song http://gerriksonger.wordpress.com/
Glen Brown Teacher Poet Musician http://teacherpoetmusicianglenbrown.blogspot.com/
Good Morning Art Teacher Good Morning Art Teacher http://goodmorningartteacher.blogspot.com/
Greg Mild Plumberbund http://www.plunderbund.com/
Guy Brandenburg Guy Brandenburg https://gfbrandenburg.wordpress.com/
Helen Gym Philadelphia Public School Notebook http://thenotebook.org/blog
Jack McKay Horace Mann League Blog http://blog.hmleague.org/
James Arnold Dr. James Arnold http://drjamesarnold.blogspot.com/
James Avington Miller, Jr The War Report on Public Education http://thewarreportonpubliceducation.wordpress.com and http://bbsradio.com/thewarreport
James Boutin An Urban Teachers Education http://www.anurbanteacherseducation.com/
James Chascherrie Stop Common Core in Washington State http://stopcommoncorewa.wordpress.com/
James Hamric Hammy’s Education Blog http://edreformblog.wordpress.com/
Jan Resseger Jan Resseger http://janresseger.wordpress.com/
Jane Nixon Willis Staying Strong in School http://stayingstronginschool.blogspot.com/
Jason France Crazy Crawfish http://crazycrawfish.wordpress.com/
Jason L. Endacott EduSanity http://www.edusanity.com/
Jason Stanford Jason Stanford http://www.huffingtonpost.com/jason-stanford/
Jeff Bryant Jeff Bryant http://educationopportunitynetwork.org/
Jen Hogue V.A.M. It! http://valueaddedmeasureit.blogspot.com/
Jennifer Berkshire EduShyster http://edushyster.com/
Jesse Hagopian Jesse Hagopian http://iamaneducator.com/
Jessie Ramey Yinzercation http://yinzercation.wordpress.com/
Jill Conroy The Indignant Teacher http://theindignantteacher.wordpress.com/
Jo Lieb Poetic Justice http://poeticjusticect.com/
Joe Bower For the love of learning http://www.joebower.org/
John J. Viall A Teacher on Teaching http://ateacheronteaching.blogspot.com/
John Kuhn EdGator https://edgator.com
John Young Transparent Christina http://transparentchristina.wordpress.com/
Jonathan Lovell Jonathan Lovell’s Blog http://jonathanlovell.blogspot.com/
Jonathan Pelto Wait, What? http://jonathanpelto.com/
Jose Vilson Jose Vilson http://thejosevilson.com/
Joshua Block Joshua Block http://mrjblock.com/
Julian Vasquez Heilig Cloaking Inquity http://cloakinginequity.com/
Justin Aion Relearning to Teach http://relearningtoteach.blogspot.com/
Karren Harper Royal Edutalknola http://edutalknola.com/
Katie Lapham Critical Classrooms https://criticalclassrooms.wordpress.com/
Ken Derstine Defend Public Education http://www.defendpubliceducation.net/
Ken Previti Reclaim Reform http://reclaimreform.com/
Kenneth Bernstein Teacher Ken http://www.dailykos.com/user/teacherken
Kevin Welner Kevin Welner http://www.huffingtonpost.com/kevin-welner/ andhttp://nepc.colorado.edu
Lani Cox The Missing Teacher http://lanivcox.blogspot.com/
Larry Cuban Larry Cuban http://larrycuban.wordpress.com/
Larry Feinberg Keystone State Education Coalition http://keystonestateeducationcoalition.blogspot.com/
Lee Barrios Geauxteacher http://www.geauxteacher.net/
Leonard Isenberg Perdaily http://www.perdaily.com/
Leonie Haimson Class Size Matters http://nycpublicschoolparents.blogspot.com/
Levi B Cavener Idahospromise http://idahospromise.org/
Linda Thomas Restore Reason http://restorereason.com/
Lisa Guisbond Fairtest http://www.fairtest.org/
Lloyd Lofthouse Crazy Normal the classroom expose http://crazynormaltheclassroomexpose.com/  or http://lloydlofthouse.org/
Lucianna Sanson The War Report on Public Education https://thewarreportonpubliceducation.wordpress.com/
M. Shannon Hernandez My Final 40 Days http://myfinal40days.com/
Maria Rosa THE INSURGENT TEACHER BLOG http://theinsurgentteacher.blogspot.com/
Marie Corfield Marie Corfield http://mcorfield.blogspot.com/
Marion Brady Marion Brady http://www.marionbrady.com/
Mark Naison With a Brooklyn Accent and Dump Duncan http://withabrooklynaccent.blogspot.com/ and http://dumpduncan.org/
Mark Weber Jersey Jazzman http://jerseyjazzman.blogspot.com/
Martha Infante Martha Infante http://dontforgetsouthcentral.blogspot.com/
Matt Farmer Matt Farmer http://www.huffingtonpost.com/matt-farmer/
Mel Katz The Education Activist: From Student to Teacher https://theeducationactivist.wordpress.com/
Melissa Westbrook Seattle Schools Community Forum http://saveseattleschools.blogspot.com/
Mercedes Schneider Deutsch29 http://deutsch29.wordpress.com/
Michael Klonsky Michael Klonsky http://michaelklonsky.blogspot.com/ and http://schoolingintheownershipsociety.blogspot.com/
Michelle Gunderson Education Matters https://www.facebook.com/michelle.gunderson.education.matters
Mike Deshotels Louisiana Educator http://louisianaeducator.blogspot.com/
Mike Rose Mike Rose’s Blog http://mikerosebooks.blogspot.com
Mike Warner Education Under Attack http://educationunderattack.info/
Minnsanity Minnsanity http://minnsanity.wordpress.com/
Morna McDermott Education Alchemy http://www.educationalchemy.com/
Mrs. Fanning LA Woman http://fanninglawoman.blogspot.com/
Ms Kate Ms Katie’s Ramblings http://mskatiesramblings.blogspot.com/
Nancy Bailey Nancy Bailey’s Education Website http://nancyebailey.com/
Nancy Flanagan Teacher in a Strange Land http://blogs.edweek.org/teachers/teacher_in_a_strange_land/
Nicholas Tampio Nicholas Tampio http://www.huffingtonpost.com/nicholas-tampio/
Nikhil Goyal Nikhil Goyal http://nikhilgoyal.me/
Norm Scott Ed Notes Online http://ednotesonline.blogspot.com/
Ogo Okoye-Johnson Ogo Okoye-Johnson http://ogookoye-johnson.net/
OK Education Truth okeducationtruths http://okeducationtruths.wordpress.com/
Outside The Box Outside the Box http://teacher-anon.blogspot.com/ 
Patrick Walsh http://raginghorse.wordpress.com/
Paul Horton Education News http://www.educationviews.org/author/paulh/
Paul Thomas The becoming radical http://radicalscholarship.wordpress.com/
Peggy Robertson Peg with Pen http://www.pegwithpen.com/
Perdido St School Perdido St School http://perdidostreetschool.blogspot.com/
Peter DeWitt Peter DeWitt http://blogs.edweek.org/edweek/finding_common_ground/
Peter Goodman Ed in the Apple http://mets2006.wordpress.com/
Peter Greene Curmudgucation http://www.curmudgucation.blogspot.com/
Phillip Cantor Sustainable Education Transformation http://phillipcantor.com/
Rachael Stickland Student Privacy Matters http://www.studentprivacymatters.org/
Rachel Levy All Things Education http://allthingsedu.blogspot.com/
Ralph Ratto Opine I will http://rlratto.wordpress.com/
Ray Salazar The White Rino http://www.chicagonow.com/white-rhino
Rob Miller View From the Edge http://www.viewfromtheedge.net/
Rob Panning-Miller Public Education Justice Alliance of Minnesota http://pejamn.blogspot.com/
Robert Cotto Jr. The Cities, Suburbs & Schools Project http://commons.trincoll.edu/cssp/
Robert D. Skeels Solidaridad http://rdsathene.blogspot.com/
Russ Walsh Russ on Reading http://russonreading.blogspot.com/
Ruth Conniff Public School Shakedown http://www.publicschoolshakedown.org/
Sam Chaltain Sam Chaltain http://www.samchaltain.com
Sara Roos Sara Roos http://redqueeninla.com/
Sarah Blaine Parenting the core http://parentingthecore.wordpress.com/
Sarah Darer Littman Sarah Darer Littman http://www.ctnewsjunkie.com
Sarah Lahm Sarah Lahm http://www.tcdailyplanet.net/eyes-education
Save Public Education Save Public Education
Sharon Higgins Charter School Scandals http://charterschoolscandals.blogspot.com/
Shaun Johnson Chalk Face http://atthechalkface.com/
Sherman Dorn Sherman Dorn http://shermandorn.com/wordpress/
South Bronx School South Bronx School http://www.southbronxschool.com/
Stephanie Rivera Teacher Under Construction http://teacherunderconstruction.com/
Stephen Dyer 10th Period http://10thperiod.blogspot.com/
Stephen Krashen Stephen Krashen http://www.schoolsmatter.info/ and http://skrashen.blogspot.com/
Steve Hinnefeld Steve Hinnefeld http://inschoolmatters.wordpress.com/
Steve O’Donoghue Steve O’Donogue http://www.counterintuitive.com/
Steve Strieker One Teachers Perspective http://oneteachersperspective.blogspot.com/
Steven Singer Gad Fly On the Wall Blog http://gadflyonthewallblog.wordpress.com/
Stu Bloom Live Long and Prsoper http://bloom-at.blogspot.com/
Sullio The Pen is Mightier than the Person http://sullio.blogspot.com/
Susan DuFresne Educating the Gates Foundation http://educatingthegatesfoundation.com/
Susan DuFresne and Katie Lapham Teachers Letters to Bill Gates http://teachersletterstobillgates.com/
Susan Ohanian Susan Ohanian http://www.susanohanian.org/
TB Furman tbfurman http://www.tbfurman.us/
TC Dad Gone Wild http://norinrad10.wordpress.com/
Teacher Reality Teacher Reality http://teacherreality.com/
Teacher Tom Teacher Tom http://teachertomsblog.blogspot.com/
Ted Cohen Newark Schools For Sale http://NewarkSchoolsForSale.wordpress.com
The Assailed Teacher http://theassailedteacher.com/
The Teaching Nomad The Teaching Nomad www.theteachingnomad.com/blog 
Tim Slekar Busted Pencils http://bustedpencils.com/ 
Tom Aswell Louisiana Voice http://louisianavoice.com/
Tracy Novick Who-cester Blog http://who-cester.blogspot.com/
Ty Alper Ty Alper (SF School Board candidate) http://www.huffingtonpost.com/ty-alper/ or http://www.tyalper.org
Urban Ed Urban Ed http://nycurbaned.blogspot.com/
Vanessa Vaile Precarious Faculty Blog http://www.precariousfacultyblog.com/ or http://nationalmobilizationforequity.org/
Wag the Dog Wag the Dog http://vigornotrigor.wordpress.com/
Walt Gardner Walt Garnder http://blogs.edweek.org/edweek/walt_gardners_reality_check/
Wayne Gersen Network Schools http://waynegersen.com/
Wendy Lecker Wendy Lecker http://www.stamfordadvocate.com
Xian Barrett Xian Barrett http://newvoicestrategies.org/
Yohuru Williams Yohuru Williams http://www.yohuruwilliams.net/
Yong Zhao Education in the Age of Globalization http://zhaolearning.com

The Real Lesson of Singapore Math!

By now you’ve probably heard that Singapore and Shanghai are the two places on earth with the smartest kids in the entire world. We can see their PISA scores (go to page 5) are right at the top.

Case closed, right? Whatever they are doing in education, we in the US need to emulate that in order to catch up! Common Core! StudentsFirst! Teach for America! Race to the Top! PARCC! Bust those teacher unions! No more recess! All test prep all the time! Charter Schools! Turn the schools over to the billionaires (Gates, Bloomberg, Koch family, Walton family, and their hirelings and shills)!

But wait a second.

Have you noticed that an ENORMOUS fraction of the low-skilled, low-paid people living in Singapore are temporary foreign workers from various parts of Asia and Africa and are not allowed to bring their kids with them? Those kids are raised back in the workers’ homelands by various relatives, far away, and only get to see their parents at long intervals (somebody has to fly somewhere); back home, jobs are even scarcer and worse-paid, so the parents go elsewhere to try support their families.

Now, everywhere in the world, family income is very, very closely linked to children’s test scores in school. It’s one of the tightest correlations there are in the social sciences, as you can see in the simple scatter-plots I have repeatedly shown in this blog over the past 4 or 5 years. (Try using terms like “poverty” “income” and “scores” together in the search box on this page and be prepared to look through a lot of posts with such graphs, from all over!)

If one-quarter to one-third of the population of a country was legally not permitted to have children in the schools, and it was the low-paying 1/4 to 1/3 of the population, then the scores of the remainder of the kids would, quite naturally, be pretty darned good, since the bottom 1/4 to 1/3 of the distribution just got cut off.

If we systematically excluded the poorest quarter or third of our American student population from taking PISA, we know that our scores would be pretty darned high as well.*

Hmm, maybe the leaning tower of PISA hype is falling.

 

=====================

*Let’s remember that this WAS official policy in many states of the USA up until 1865: a large fraction of the population (guess which one!) was forbidden to send their kids to schools at all and it was explicitly forbidden even to teach them to read privately. When Jim Crow was established from the 1870s to the early 1960s, school facilities for Blacks and Hispanics, BY DESIGN of the racist authorities, so inferior to those for whites that they were a national disgrace. Which is why the calls for going back to the good old days should be so infuriating. There WERE NO GOOD OLD DAYS.

What I actually had time to say …

Since I had to abbreviate my remarks, here is what I actually said:

I am Guy Brandenburg, retired DCPS mathematics teacher.

To depart from my text, I want to start by proposing a solution: look hard at the collaborative assessment model being used a few miles away in Montgomery County [MD] and follow the advice of Edwards Deming.

Even though I personally retired before [the establishment of the] IMPACT [teacher evaluation system], I want to use statistics and graphs to show that the Value-Added measurements that are used to evaluate teachers are unreliable, invalid, and do not help teachers improve instruction. To the contrary: IVA measurements are driving a number of excellent, veteran teachers to resign or be fired from DCPS to go elsewhere.

Celebrated mathematician John Ewing says that VAM is “mathematical intimidation” and a “modern, mathematical version of the Emperor’s New Clothes.”

I agree.

One of my colleagues was able to pry the value-added formula [used in DC] from [DC data honcho] Jason Kamras after SIX MONTHS of back-and-forth emails. [Here it is:]

value added formula for dcps - in mathtype format

One problem with that formula is that nobody outside a small group of highly-paid consultants has any idea what are the values of any of those variables.

In not a single case has the [DCPS] Office of Data and Accountability sat down with a teacher and explained, in detail, exactly how a teacher’s score is calculated, student by student and class by class.

Nor has that office shared that data with the Washington Teachers’ Union.

I would ask you, Mr. Catania, to ask the Office of Data and Accountability to share with the WTU all IMPACT scores for every single teacher, including all the sub-scores, for every single class a teacher has.

Now let’s look at some statistics.

My first graph is completely random data points that I had Excel make up for me [and plot as x-y pairs].

pic 3 - completely random points

Notice that even though these are completely random, Excel still found a small correlation: r-squared was about 0.08 and r was about 29%.

Now let’s look at a very strong case of negative correlation in the real world: poverty rates and student achievement in Nebraska:

pic  4 - nebraska poverty vs achievement

The next graph is for the same sort of thing in Wisconsin:

pic 5 - wisconsin poverty vs achievement

Again, quite a strong correlation, just as we see here in Washington, DC:

pic 6 - poverty vs proficiency in DC

Now, how about those Value-Added scores? Do they correlate with classroom observations?

Mostly, we don’t know, because the data is kept secret. However, someone leaked to me the IVA and classroom observation scores for [DCPS in] SY 2009-10, and I plotted them [as you can see below].

pic 7 - VAM versus TLF in DC IMPACT 2009-10

I would say this looks pretty much no correlation at all. It certainly gives teachers no assistance on what to improve in order to help their students learn better.

And how stable are Value-Added measurements [in DCPS] over time? Unfortunately, since DCPS keeps all the data hidden, we don’t know how stable these scores are here. However, the New York Times leaked the value-added data for NYC teachers for several years, and we can look at those scores to [find out]. Here is one such graph [showing how the same teachers, in the same schools, scored in 2008-9 versus 2009-10]:

pic 8 - value added for 2 successive years Rubenstein NYC

That is very close to random.

How about teachers who teach the same subject to two different grade levels, say, fourth-grade math and fifth-grade math? Again, random points:

pic 9 - VAM for same subject different grades NYC rubenstein

One last point:

Mayor Gray and chancellors Henderson and Rhee all claim that education in DC only started improving after mayoral control of the schools, starting in 2007. Look for yourself [in the next two graphs].

pic 11 - naep 8th grade math avge scale scores since 1990 many states incl dc

 

pic 12 naep 4th grade reading scale scores since 1993 many states incl dc

Notice that gains began almost 20 years ago, long before mayoral control or chancellors Rhee and Henderson, long before IMPACT.

To repeat, I suggest that we throw out IMPACT and look hard at the ideas of Edwards Deming and the assessment models used in Montgomery County.

Poverty Isn’t Destiny?

Quite a few Ed Deformers say that Poverty Isn’t Destiny. They say that it doesn’t matter if a child has been subjected to lead poisoning, separation from parents, violent or otherwise cruel child abuse, inadequate nutrition, and has lacked dental or health care and the love and care of a family during the first, crucial years. All it takes is for a Bright Young Thing fresh out of college to work her butt off for two years before she goes to work for a bank — and all of those handicaps will be overcome, with no extra dollars invested, and maybe even less!

Or maybe not.

Lots of teachers have been working their butts off for many decades, doing their best, believe it or not (for the most part).

Here are two three graphs from Wisconsin that show how close the connection between the poverty rates and student achievement levels, at all of their schools for which they provide data. My data come from here and are for SY 2011-2012. In fact, you can download the entire spreadsheet for the state of Wisconsin if you click on this link:

http:reportcards.dpi.wi.gov/files/reportcards/xls/2011-12reportcarddata.xlsx

In both all three graphs, the percentage of students at the schools is along the horizontal (X) axis. In the first two, the average achievement score at the school is along the vertical (Y) axis.

In this first graph, Wisconsin uses a 100-point scale for overall student achievement.

wisconsin school overall student ach score by pct of poor kids

That is an incredibly strong correlation between poverty levels and student achievement. The fewer the proportion of poor students at a school, the better the achievement scores at that school.

I had Excel compute two correlation “trend” lines – one straight, in black, and one curved, in red following a third-degree polynomial, since it looks like we have a serious “Matthew effect” going on here. In either case, the R-squared and R values are very elevated, showing that, in fact, poverty is in fact destiny for a lot of kids.

The next graph is for reading only, but it shows essentially the same trend. School reading scores go from 0 to 50.

Wisconsin school READING scores by pct of poor kids

There are very few real-life correlations between two entities stronger than what you see in these two graphs.

This next graph is a little different, for two reasons: the y-axis is math, and it’s the percent of students deemed ‘proficient’ on whatever test Wisconsin is using. It also shows a very strong correlation.

wisconsin school poverty rate versus percent of students proficient in MATH

Teacher VAM scores aren’t stable over time in Florida, either

I quote from a paper studying whether value-added scores for the same teachers tend to be consistent. In other words, does VAM allow us a chance to pick out the crappy teachers and give bonuses to the good one?

The answer, in complicated language, is essentially NO, but here is how they word it:

“Recently, a number of school districts have begun using measures of teachers’ contributions to student test scores or teacher “value added” to determine salaries and other monetary rewards.

In this paper we investigate the precision of valueadded measures by analyzing their inter-temporal stability.

We find that these measures of teacher productivity are only moderately stable over time, with year-to-year correlations in the range of 0.2-0.3.”

Or in plain English, and if you know anything at all about scatter plots and linear correlation, those scores wander all over the place and should never be used to provide any serious evidence about anything. Speculation, perhaps, but not policy or hiring or firing decisions of any sort.

They do say that they have some statistical tricks that allow them to make the correlation look better, but I don’t trust that sort of thing.  It’s not real.

Here’s a table from the paper. Look at those R values, and note that if you squared those correlation constants (go ahead, use your calculator on your cell phone) you get numbers that are way, way smaller – like what I and Gary Rubenstein reported concerning DCPS and NYCPS.

For your convenience, I circled the highest R value, 0.61, in middle schools on something called the normed FCAT-SSS, whatever that is (go ahead and look it up if it interests you) in  Duval county, Florida, one of the places where they had data. I also circled the lowest R value, 0.07, in Palm Beach  county, on the FCAT-NRT, whatever that is.

I couldn’t resist, so 0.56^2 is about 0.31 as an r-squared, which is moderate. There is only one score anywhere near that high 0.56, out of 24 such correlation calculations. The lowest value is 0.07 and if we square that and round it off we get an r-squared value of 0.005, shockingly low — essentially none at all.

The median correlation constant is about 0.285, which I indicated by circling two adjacent values of 0.28 and 0.29 in green. If you square that value you get r^2=0.08, which is nearly useless. Again.

I’m really sorry, but even though this paper was published four years ago, it’s still under wraps, or so it says?!?! I’m not supposed to quote from it? Well, to hell with that. it’s important data, for keereissake!

The title and authors are as follows, and perhaps they can forgive me. I don’t know how to contact them anyway. Does anybody have their contact information? Here is the title, credits, and warning:

THE INTERTEMPORAL STABILITY OF TEACHER EFFECT ESTIMATES *
by
Daniel F. McCaffrey;         Tim R. Sass;                         J. R. Lockwood
The RAND Corporation; Florida State University; The RAND Corporation
Original Version: April 9, 2008
This Version: June 27, 2008

*This paper has not been formally reviewed and should not be cited, quoted, reproduced, or retransmitted without the authors’ permission. This material is based on work supported by a supplemental grant to the National Center for Performance Initiatives funded by the United States Department of Education, Institute of Education Sciences. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of these organizations.

Published in: on March 30, 2012 at 3:58 pm  Comments (5)  
Tags: , ,

The Correlation Between ‘Value-Added’ Scores and Observation Scores in DCPS under IMPACT is, in fact, Exceedingly Weak

As I suspected, there is nearly no correlation between the scores obtained by DCPS teachers on two critical measures.

I know this because someone leaked me a copy of the entire summary spreadsheet, which I will post on the web at Google Docs shortly.

As usual, a scatter plot does an excellent job of showing how ridiculous the entire IMPACT evaluation system is. It doesn’t predict anything to speak of.

Here is the first graph.

Notice that the r^2 value is quite low: 0.1233, or about 12%. Not quite a random distribution, but fairly close. Certainly not something that should be used to decide whether someone gets to keep their job or earn a bonus.

The Aspen Institute study apparently used R rather than r*2; they reported R values of about 0.35, which is about what you get when you take the square root of 0.1233.

Here is the second graph, which plots teachers’ ranks on the classroom observations versus their ranks on the Value Added scores. Do you see any correlation?

 

Remember, this is the correlation that Jason Kamras said was quite strong.

More Value Added Comparisons

Someone who professes to understand Value-Added scores better than me claims that my graphs for NYC are meaningless because the scores for 2007 were inflated; he claimed that the overall year-to-year and year-to-career value-added correlation coefficients are much higher than what I found — thus, VA is really useful, just not my particular graphs..

Taking this objection seriously, I decided to leave out SY 0607, and compare SY 0506 to SY 0708. Same exact teachers, same exact subjects and grade levels, same exact schools, obviously different (but quite similar) kids.

Here is the scatterplot of what I found. Again, I asked Excel to calculate a line of best fit, and it drew it. Notice that the r-squared correlation value is about 0.05 — seriously LOW. Notice also that this scatterplot is basically a blob again, again a classic example of one variable showing very little correlation with another. (West Virginia’s map has a much more defined shape!) In any case, there are lots (hundreds? thousands?) of teachers with positive VA scores in the first year and negative VA scores the third year, and vice-versa. Only an easily countable handful of teachers have scores of +0.2 or better both years, or worse than -0.2 both years. Out of all of the thousands of teachers. And I bet those are all accidents as well.

So, in other words, I find, as did Gary Rubenstein, that there is extremely little correlation between two things that should be, you would think, very close to a perfect 1.00 correlation. (In the real world, of course, you almost never get a 1.00 correlation between any real entities or quantities. However, when you are talking about the scores of teachers who have been teaching IN THE SAME SCHOOL, THE SAME SUBJECT, THE SAME GRADE LEVEL for three straight years, then you would think that their performances would be rather similar all three years. If anything, they would normally get better unless they had suffered some sort of physically or mentally debilitating injury or illness (often from old age and the incredible amount of stress). In particular, a lot of teachers will admit to you that they absolutely sucked at teaching during their first year, but that they then figured out a lot of those errors and tried not to make the same ones the next year, so they really improved, or else they quit. But these folks didn’t quit. These are at the very least three-year veterans, which in DC would make them eligibility for department or grade level chair at their school as a result of seniority alone, since so many of the older teachers have quit or retired, and the turnover and attrition over the last few years among the newest hires in our school system is probably unprecedented in the history of education. (Perhaps not, but it’s a subject I’d like to pursue.)

 

Finally, while I admit that I exaggerated a bit (for effect) when I said that the shapes of these graphs, and the very low computed values for the r-squared coefficient of linear correlation, made value-added about as predictive as numerology. I thought about that particular exaggeration and wondered how serious it was. So, even though I have participated in a fairly large number of courses on calculating probabilities and distribution, it’s always a bit fraught with error: Have we counted all of the possibilities? Have we left any out? Have we double-counted any of them? Is there a much better, faster, or less error-prone method hidden right around the corner?

 

To make a long story short: the Monte Carlo method is a great way of deciding, say, how likely something is to happen. It’s called “Monte Carlo” because it’s very much like gambling in a casino, except you a4ren’t betting any5thing except your time. You just roll some dice (they might be funny-looking non-cubical polyhedra) or spinning a wheel or throwing darts or spattering paint or vaporized metal… And then you see what happens, and draw conclusions. Today, it’s 4really easy t6o do.

So I decided to see whether, in fact, the number of letters in the teachers’ names had any correlation with their Value Added scores. (I thought it was possible, tho not very likely.) I discovered that Excel found the r-squared constant was about 0.000000. That is zero correlation, my friends. Here is one such scatterplot:

The vertical axis, which goes up the middle, is the number of letter in the teachers’ first name times the number of letters in their last name as listed in the spreadsheet. The horizontal axis, which is at the bottom of the page, is their 2005-2006 value-added score, which can be either negative (theoretically bad) or positive (supposedly good). To me, it sort of looks like bush that hasn’t been pruned in several years – a classic case of no correlation at all.

I asked Excel to draw and calculate the line of best fit. It’s the green, nearly-horizontal line near the center of the graph. Notice the r-squared value: 6E-05, which for all of you innumerates out there, means 0.00006, which is seriously smaller (three orders of magnitude smaller) than 0.05; i.e., one-thousandth as big.

Notice that I’m only using r-squared. Someone objected that i should use just r. If you want, take the square root of all of the correlations I had my computer calculate, and you’ll get r. Compare and contrast.

So, in any case, I definitely did exaggerate.

Whether DC-CAS scores go up or down at any school seems mostly to be random!

After reviewing the changes in math and reading scores at all DC public schools for 2006 through 2009, I have come to the conclusion that the year-to-year school-wide changes in those scores are essentially random. That is to say, any growth (or slippage) from one year to the next is not very likely to be repeated the next year.

Actually, it’s even worse than that.The record shows that any change from year 1 to year 2 is somewhat NEGATIVELY correlated to the changes between year 2 and year 3. That is, if there is growth from year 1 to year 2, then, it is a bit more likely than not that there will be a shrinkage between year 2 and year 3.  Or, if the scores got worse from year 1 to year 2, they there is a slightly better-than-even chance that the scores will improve the following year.

And it doesn’t seem to matter whether the same principal is kept during all three years, or whether the principals are replaced one or more times over the three-year period.

In other words, all this shuffling of principals (and teachers) and turning the entire school year into preparation for the DC-CAS seems to be futile. EVEN IF YOU BELIEVE THAT THE SOLE PURPOSE OF EDUCATION IS TO PRODUCE HIGH STANDARDIZED TEST SCORES. (Which I don’t.)

Don’t believe me? I have prepared some scatterplots, below, and you can see the raw data here as a Google Doc.

My first graph is a scatterplot relating the changes in percentages of students scoring ‘proficient’ or better on the reading tests from Spring 2006 to Spring 2007 on the x-axis, with changes in percentages of students scoring ‘proficient’ or better in reading from ’07 to ’08 on the y-axis, at DC Public Schools that kept the same principals for 2005 through 2008.

If there were a positive correlation between the two time intervals in question, then the scores would cluster mostly in the first and third quadrants. And that would mean that if scores grew from ’06 to ’07 then they also grew from ’07 to ’08; or if they went down from ’06 to ’07, then they also declined from ’07 to ’08.

But that’s not what happened. In fact, in the 3rd quadrant, I only see one school – apparently  M.C.Terrell – where the scores went down during both intervals. However, there are about as many schools in the second quadrant as in the first quadrant. Being in the second quadrant means that the scores declined from ’06 to ’07 but then rose from ’07 to ’08. And there appear to be about 7 schools in the fourth quadrant. Those are schools where the scores rose from ’06 to ’07 but then declined from ’07 to ’08.

I asked Excel to calculate a regression line of best fit between the two sets of data, and it produced the line that you see, slanted downwards to the right. Notice that R-squared is 0.1998, which is rather weak. If we look at R, the square root of R-squared, that’s the regression constant, my calculator gives me -0.447, which means again that the correlation between the growth (or decline) from ’06 to ’07 is negatively correlated to the growth (or decline) from ’07 to ’08 – but not in a strong manner.

OK. Well, how about during years ’07-’08-’09? Maybe Michelle Rhee was better at picking winners and losers than former Superintendent Janey? Let’s take a look at schools where she allowed the same principal to stay in place for ’07, ’08, and ’09:

Actually, this graph looks worse! There are nearly twice as many schools in quadrant four as in quadrant one! That means that there are lots of schools where reading scores went up between ’07 and ’08, but DECLINED from ’08 to ’09; but many fewer schools where the scores went up both years. In the second quadrant, I  see about four schools where the scores declined from ’07 to ’08 but then went up between ’08 and ’09. Excel again provided a linear regression line of best fit, and again, the line slants down and to the right. R-squared is 0.1575, which is low. R itself is about -0.397, which is, again, rather low.

OK, what about schools where a principal got replaced? If you believe that all veteran administrators are bad and need to be replaced with new ones with limited or no experience, you might expect to see negative correlations, but with positive overall outcomes; in other words, the scores should cluster in the second quadrant. Let’s see if that’s true. First, reading changes over the period 2006-’07-’08:

Although there are schools in the second quadrant, there are also a lot in the first quadrant, and I also see more schools in quadrants 3 and 4 than we’ve seen in the first two graphs. According to Excel, R-squared is extremely low: 0.0504, which means that R is about -0.224, which means, essentially, that it is almost impossible to predict what the changes would be from one year to the next.

Well, how about the period ’07-’08-’09? Maybe Rhee did a better job of changing principals then? Let’s see:

Nope. Once again, it looks like there are as many schools in quadrant 4 as in quadrant 1, and considerably fewer in quadrant 2. (To refresh your memory: if a school is in quadrant 2, then the scores went down from ’07 to ’08, but increased from ’08 to ’09. That would represent a successful ‘bet’ by the superintendent or chancellor. However, if a school is in quadrant 4, that means that reading scores went up from ’07 to ’08, but went DOWN from ’08 to ’09; that would represent a losing ‘bet’ by the person in charge.) Once again, the line of regression slants down and to the right.  The value of R-squared, 0.3115, is higher than in any previous scatterplot (I get R = -0.558) which is not a good sign if you believe that superintendents and chancellors can read the future.

Perhaps things are more predictable with mathematics scores? Let’s take a look. First, changes in math scores during ’06-’07-’08 at schools that kept the same principal all 3 years:

Doesn’t look all that different from our first Reading graph, does it? Now, math score changes during ’07-’08-’09, schools with the same principal all 3 years:

Again, a weak negative correlation. OK, what about schools where the principals changed at least once? First look at ’06-’07-‘-8:

And how about ’07-’08-’09 for schools with at least one principal change?

Again, a very weak negative correlation, with plenty of ‘losing bets’.

Notice that every single one of these graphs presented a weak negative correlation, with plenty of what I am calling “losing bets” – by which I mean cases where the scores went up from the first year to the second, but then went down from the second year to the third.

OK. Perhaps it’s not enough to change principals once every 3 or 4 years. Perhaps it’s best to do it every year or two? (Anybody who has actually been in a school knows that when the principal gets replaced frequently, then it’s generally a very bad sign. But let’s leave common sense aside for a moment.) Here we have scatterplots showing what the situation was, in reading and math, from ’07 through ’09, at schools that had 2 or more principal changes from ’06 to ’09:

and

This conclusion is not going to win me lots of friends among those who want to use “data-based” methods of deciding whether teachers or administrators keep their jobs, or how much they get paid. But facts are facts.

==============================================================================

A little bit of mathematical background on statistics:

Statisticians say that two quantities (let’s call them A and B) are positively correlated when an increase in one quantity (A)  is linked to an increase in the other quantity (B). An example might be a person’s height(for quantity A) and length of a person’s foot (for quantity B). Generally, the taller you are, the longer your feet are. Yes, there are exceptions, so these two things don’t have a perfect correlation, but the connection is pretty strong.

If two things are negatively correlated, that means that when one quantity (A) increases, then the other quantity (B) decreases. An example would be the speed of a runner versus the time it takes to run a given distance.  The higher the speed at which the athlete runs, the less time it takes to finish the race. And if you run at a lower speed, then it takes you more time to finish.

And, of course, there are things that have no correlation to speak of.

Published in: on March 13, 2010 at 3:37 pm  Comments (2)  
Tags: , , , , ,
Follow

Get every new post delivered to your Inbox.

Join 454 other followers

%d bloggers like this: