Here are two graphs that I made concerning the correlation, or lack thereof, between general indicators of poverty and student achievement.
One is from DC public elementary schools, and one is from DC charter schools.
I used the NCLB-OSSE data to look up the numbers of students deemed ‘economically disadvantaged’ at each regular DC public school, and then the total population tested at that school. From that I calculated the percent of students in poverty. I got the percent of students ‘passing’ in reading on the DC-CAS for 2010 simply by adding the percentages of students scoring in the ‘proficient’ or ‘advanced’ categories. I then plotted the poverty rate versus the passing rate, and commanded Excel to draw a line of best fit.
The overall trend is that the higher the percentage you have of poor students, in general, the lower the achievement scores, with a R-squared value of 0.6555. But the correlation is nowhere near as strong as what we saw with the data from DC as a whole BY WARD. Furthermore, it seems to me as if the elementary schools in DCPS are really split into two categories: the ones with very few kids on free or reduced-price lunch (that is, the 13 or so schools with less than 45% poverty rates) on the one hand, and the 70 or so schools with over 60% poverty rates on the other hand.
In fact, we have 57 schools (about 68% of the total number of schools) in DCPS with poverty rates greater than 80%. These are the ones that fill the two right-hand blocks on the graph. Only a small handful of those very-high-poverty schools have ‘pass’ rates that are greater than 50%.
I will list the 13 DC public schools with low poverty rates, and I bet that many of my readers could guess the names of most of them. Here is the list, which represents about 15% of all regular DC public elementary schools. Next to each school is the percentage of students in poverty.
My next exhibit is a similar graph for the forty or so DC public elementary CHARTER schools.
As you can see, the correlation between poverty and achievement is not nearly as strong in the charter schools as it is in the regular public schools. In other words, the dots don’t all cluster so tightly around the line of best fit, and the R-squared value is much lower, namely, 0.1646. Furthermore, there is not nearly the same clustering of high-poverty schools as we see in the regular DC schools. If you recall, DC public elementary schools overwhelmingly had greater than 80% poverty rates, and almost none between 40% and 60% poverty rates. Not so the charter schools, of which I count a total of 42. I see a relatively small group of schools with low poverty rates, which I will list:
Childrens Studio School 9%
Achievement Preparatory Academy 11%
NIA Community 20%
Ideal Academy Pcs – North Capitol Street Campus 23%
Latin American Montessori Bilingual 30%
Two Rivers – Elementary 30%
Community Academy Pcs – Online Program 30%
That group represents about 19% of all DC elementary charter schools.
The intermediate group, which has between 40 and 60% poverty rates, consists of nine schools (21% of the entire group – as opposed to a single, solitary DC public school). There are 26 charter schools (62% of the total) which have greater than 60% poverty rates. And there are only 16 public scharter schools (38% of the total) which have greater than 80% poverty rates.
So this graph appears to confirm what I suspected (and wrote) earlier: middle-income students seem to be deserting the regular public schools and flocking to the charter schools. A lot of the very poorest students are remaining behind in the most impoverished public schools.
————-
OK, how about achievement, as measured by the DC-CAS? (Not that I think it’s a very good test, but it’s all we have to go on.)
Well, in the regular public schools, we see that there are 20 schools (about 24% of the total number of schools) which have greater than 50% ‘passing’ rates in reading. In the charter schools, there are 12 schools (about 29% of the total) which have greater than 50% ‘passing’ rates in reading.
———–
Extremely alert readers may wonder where the scores for Sharpe Health, St. Coletta, the Hamilton Center, and the Mamie D. Lee school are located. I felt sufficiently skeptical about those scores not to include them at all.
——————
IMPORTANT NOTE:
My first version of this column had a graph that alleged to be data from Nebraska public school districts. It fit a line of negative correlation so well that it was just as scary as the graph produced by Allan Assarsson and his son. I attempted to check the data by looking up at least a handful of the achievement scores for the Nebraska school districts; the data I found all agreed with the data I was given. However, I kinda wondered if the entire state of Nebraska really only had twenty (20) school districts. That number seemed a little low (not that I know a lot about Nebraska), and the data seemed to fit the line too well.
Today I decided to look up a list of Nebraska school districts. It turns out that there are over 270 school districts in that state. Granted, a whole lot of them are districts that have less than 100,000 total population (not students, but all men, women, and children together). Not knowing where to find the necessary data on all 270 of those school districts, and not knowing if the author of the table had simply taken a true random sample of the districts, or had merely cherry-picked the ones that best fit his or her prior opinions, I decided to eliminate the graph and table all together.
GFBrandenburg's Blog 
Great post. Thank you.
Question for you, are the poverty rates for the entire school, or the percentage reported on the NCLB report cards on the OSSE website?
On Capitol Hill, the march of gentrification has managed to cluster all the poverty in the upper grades. For instance, Brent Elementary is no longer a Title I school, but over 50% of the children who take the test are living in poverty. Does that make sense?
I would suspect that the rate of poverty would also be much higher in the testing grades at some of the NW schools as well, since many of those parents head to private around third grade.
Just curious.
LikeLike
Those poverty rates are the ones reported on the NCLB-OSSE website for the students who are in the grades that are subject to the DC-CAS, namely grades 3 through 8 and then garde 10. So if 100% of the students in grades 1 and 2 happened to be from wealthy homes, it would not show up on my chart.
However, I doubt that gentrification happens quite that fast.
I’m considering doing a similar study for proficiency and poverty rates over time – the only problem is that it’s quite a time-consuming task, given that OSSE gives you the raw data on the total tested enrollment and on the total number in poverty (or total # hispanic, etc etc) but nowhere gives you percentages in poverty or pe4cent of the student body that is hispanic, etc. So I have to look up each one and copy the numbers, etc, etc… A royal, time-consuming, PITA.
LikeLike
Gotcha. Makes sense and fundamentally, we care about the poverty rates of the tested children.
I would love to see how that study you proposed shakes out. Look at a school like Thomson and the proficiency scores on their Asian kids. They are amazing and this is a population that is almost entirely ESL. Despite that, these children are blowing the math scores out of the water. You don’t see Thomson being held up as a shining example of testing success, but clearly something they are doing works well with a certain subset of ESL students.
LikeLike
Surprisingly enough, I have been finding that ESL students as a whole do BETTER in math than the regular DCPS population.
It may have something to do with the fact that a lot of the Asian immigrants we have here in DC are the sons and daughters of the very-best-educated people that China and Vietnam have to offer. In some cases their parents have come to America to work with their embassies or simply to study or work at a local university.
LikeLike
Mostly I think you’re right, but at Thomson most of the Asian parents are classified as low-income and work in Chinatown restaurants. (Of course that’s not to say that they aren’t highly educated in their own country and have had a change of circumstance here…)
Thomson is an interesting little case study though. Why do l Asian student at Thomson outperform all Asian students in DCPS in math, despite being low income?
LikeLike
OK, lemme modify my guess. Perhaps their parents were highly educated back home, but never learned enough English to be employable here in another field, or their skills and degrees didn’t translate into American. It’s quite had to learn a first foreign language as an adult.
Plus there is the general work-hard-and-study-harder ethic that is so strong in Chinese culture. As most of us know, hard work and dedication are what really cause ‘success’ in most fields. Having native, inborn, talent helps, but it’s hard to prove that it exists, and most of us know some very talented people (in all kinds of fields) who haven’t done squat with their lives. Many, but not all, Asian kids are force-fed that attitude from the moment they are conscious, and in many cases (not all) the attitude sticks.
BTW, I think that immigration is generally a win-win situation for the US. Folks who come to the US are generally extremely highly motivated to succeed. Heck, people are literally willing to DIE for the opportunity to come to our shores (crossing the Caribbean in tiny boats and rafts, crossing the Sonoran desert, etc etc). And we want to send them BACK because they couldn’t get the right form stamped or didn’t have the money to bribe the right officials? WTF????
LikeLike
Yes, the kids (and parents) at Thomson are some of the nicest you will meet. The Hispanic kids struggle on the DC-CAS, but too are fantastic children. Teachers do good work at Thomson. It’s a great school.
LikeLike
It is the same for Ethiopian students and some African students. At high school they tell me that the math that they do here in DCPS is equivalent to their 5th grade math, therefore they are not actually learning any math here but reviewing it throughout their years at a DCPS high shcool (whereas DC kids are learning math skills). This gives them more brain-time to focus on the language of math in the classroom. Hence they do well in math and science even with a language barrier but less well in social studies, English, etc. Math/science teachers love to have these ELLs in their classroom because it boosts their scores but not so for the ELA teachers. Also, as previously stated their parents have high expectations for their children regardless of their social economic status.
LikeLike
Any thoughts on shifting ethnic minorities. Chris Bergfaulk has been making a big deal about a declining number of black students on NAEP tests. Do you know anything about this?
LikeLike
It is true.
LikeLike
Guy –
This brings up a couple plots that would be interesting, plotting the performance of free and reduced lunch cohort against their percentage in a school. I have seen references to studies that show poor kids do better in schools with more affluent kids, just because overall school expectations are higher, and affluent parents give more to their schools. It would also be interested in which schools are outliers in a positive direction. They “may” be schools with programs worthy of study, especially if they are beating the trends over time.
I’m also curious of this data could show how much of test score increase in the past few years can be shown to be due to the change in demographics instead of any improvement in schools.
I wish the Post did half the work you do in digging deeper.
LikeLike
Guy,
Here is an evaluation of DC-CAS data showing how students did year to year as they progress thru DCPS. I have the achievement gap in math and reading, along with the number of students at each grade level. That is not necessarily the number tested.
I got the data from DC-CAS OSSE/NCLB website.
I went for State and LEA report cards, then selected DCPS as LEA, then clicked “All Schools”. When the report came up, I then clicked on “Grade levels”
Here it goes:
Washington DC
Achievement Gap of students as they progress through the school system
Results from DC-CAS
Gr Year RDG M #AA #W
Gr3 2008 44 52 2939 297
Gr4 2009 45 43 2636 285
Gr5 2010 51 54 2368 250
Gr4 2008 49 51 3010 277
Gr5 2009 49 45 2686 269
Gr6 2010 57 53 2109 215
Gr5 2008 48 50 2870 212
Gr6 2009 46 52 2109 196
Gr7 2010 51 51 2044 165
Gr6 2008 50 52 2656 181
Gr7 2009 56 50 2208 140
Gr8 2010 52 49 2208 146
Gr7 2008 56 55 2415 114
Gr8 2009 54 58 2297 146
Gr8 2008 53 60 2886 134
Gr10 2010 51 53 2124 139
Many things stand out.
There is a loss of 762 students (26%) from grade 8 (2008) to grade 10(2010)
Loss of 29% from gr 5 (08) to gr 7 (10)
16% loss gr 6 (08) to gr 8 (10)
30% loss from gr 4 (08) to gr 6 (10)
19% loss from gr3 (08) to gr 5(10)
The best Miss Rhee can claim in reducing the achievement gap is with the 8th graders of 2008.
But how much of that reduction came from the 26% decline in the number of African American students?
So, when she first came in, she said she was going to improve the achievement gap between AA and whites. Things don’t look good.
With a Fenty loss and then her exit, she can make a silk purse.
LikeLike
What I’d really like to see is a list of the high-performing high-poverty schools (the Thomson discussion has been really interesting).
Sometimes I think that in all the work of debunking (necessary and appreciated as it is), we may lose sight of what’s going right (and, more importantly, why and how/whether it can be replicated).
LikeLike
I think Guy could pick those out of this graph, though more interesting would be schools that have been high poverty and beat the core over a 5 year or longer period. They would be models to study and replicate.
Looking at this again it is interesting to note that the charters are flatter than DCPS. None of the worst performers or the best performers.
LikeLike
Yeah, but there is a caveat in all of this analysis of test scores. Namely, the tests are so worthless!!!!
LikeLike
I think your review of the test questions was great and shows the tests are weak. And I agree test scores are a shallow and narrow judgement on school and teacher performance. They do remain the only quantitative measure we have. A lot of work has to go into improving them, but as weak as they are, other measures available to us are almost completely subjective.
LikeLike
Yes, but I have learned a heck of a lot over the past few months just by sitting in the back of a whole lot of classrooms in a whole lot of schools, charter and regular. I don’t know how one could easily quantify what I’ve seen, but it’s been quite revealing to me. (Of course, IMPACT is an attempt to quantify it all by making a checklist and seeing how many of the items on the li8st are exhibited by the teacher and/or students. But it’s a stupid checklist, manufactured by mean-spirited people.) GFB
LikeLike
I’m fine with identifying high-poverty high-performing schools by means other than standardized testing. But the bottom line is that we need to look at outliers and figure out who is beating the odds and why.
That said, I think that the claim that the tests are worthless (an argument I pretty much agree with) is undercut when you treat them as measures of “achievement” in high-income schools.
NCLB has inadvertently delivered to us a teachable moment wrt that issue in Ward 3. Beyond the obvious (e.g. at a certain point you’re a victim of your own success under that system), there’s the question of whether a high-performing school would be a better school if its standardized test scores increased. And once people think that through, maybe the whole question of whether standardizes test scores measure performance or excellence is more easily re-opened.
LikeLike
[…] Here’s the thing though, if we want to be data-driven, let’s start to maybe blame poverty for bad scores instead: […]
LikeLike