The lead author of the 1995 UMBC report that analyzed the performance of the Tesseract schools recently wrote the following comment on my website on February 12, 2011 at 11:58 am:
“I was the principal investigator for the study cited by Guy Brandenburg and referenced in the Washington Post, and, yes, he has correctly presented the information therein (converting NCE scores to percentiles) and come to the appropriate conclusion that one teacher’s “90+ percentile scores” of the estimated four third-grade classrooms would have led to a higher grade average. We were evaluating the Tesseract program as a whole rather than school results by classroom, so CTBS scores available to us were not disaggregated by teacher.”
Contacted by me via a phone call for confirmation, the retired PhD researcher told me that she found it interesting that Michelle Rhee had been claiming such enormous (and unlikely) success at Harlem Park, and that she was very pleased that, at long last, people had actually begun reading the report that she and her colleagues put together. She said that back in 1995, her report had been almost completely ignored: “it fell like a stone in the water at the time.”
She also told me why there were no MSPAP scores for 1994-5 in the report: simply, that those scores would only get reported out in about October or November, but the UMBC Center for Educational Research organization that got the contract to do the report, wrote and published theirs in September of 1995. Furthermore, in 1995-6, the city of Baltimore stopped giving the CTBS and changed to a different test.
She explained to me the significance of a “1” score – which meant a student didn’t bother to fill in any answers at all, and confirmed what I remembered from that era, which is that during testing week, principals and counselors often would make arrangements to subtly discourage low-performing students from coming to school. (I also recall such students being sent to classrooms where they would watch movies, have parties, do art projects, and so on.)
Asked point-blank whether she thought my conclusions were correct, she said, “Yes.” Giving me permission to quote, she went on to say this:
“You drew, in a fair way, the conclusions that could be drawn from the information in the report, which came from the Baltimore City public tables.”
Furthermore, she added, “I don’t know how you’ll fare up against the American Enterprise Institute, but go for it!”
GFBrandenburg's Blog 
You go Guy!!! I think we are on a role here, this needs to be posted on every teaching blog website so that as many people as possible read it.
Thanks for all your hard work. A new DCPS teacher that does not believe the hype.
LikeLike
You have absolutely no idea how science/research works, do you? The opinions of the authors should NOT weigh as any evidence for or against any claims that anyone is trying to make. These opinions are rendered even more empty considering that the researcher here never actually saw the data separated at the teacher level.
LikeLike
Chris The Creationist speaks.
Rhee said, that settles it.
And to think he taught high school biology (for 2 years-the typical TFA run) at Willow Glen (where Jay Mathews’ friend cal_lanier teaches.)
Do you think Chris will honestly share the AP Bio scores of his students?
LikeLike
Will Education Week and state newspapers feature the Washington Post articles and raise additional questions about Rhee?
LikeLike
It is absolutely appropriate for the author of a research project to analyze and comment on it. In fact, the author has the most authority to do so.
In fact, it is what they are hired and obliged to do.
LikeLike
You are apparently also misinformed.
The author can only make as many conclusions about their results as their evidence merits. This person never even saw the teacher level data. Their opinions make not one difference as to how the numbers support the claims leveled.
LikeLike
Some people see evidence of god in rainbows.
Michelle Rhee is guilty of making conclusions based on hearsay and that’s okay with you?
LikeLike
Chris,
The author is commenting about a report from 1995, that’s 15 years old.
LikeLike
Your point?
LikeLike
DCPS according to Chris,
it’s 20 years ago.
He wrote this over at eduwonk.
http://www.eduwonk.com/2010/12/rhee-invented.html#comment-214370
And he complains about Guy and stats.
It seems Guy can add, atleast.
LikeLike
TFT:
That her principal and coworkers agree that they saw high performance in Rhee’s class does not indicate that Rhee never saw her numbers. You are incorrectly equating the lack of scores now, 16 years after those tests were taken, to the time when Rhee was actually in the classroom, and likely saw/heard of the large progress her students made.
LikeLike
Not only that, but if you want to cite results, the evidence we have shows gains in 3rd grade in 1995, both on this study and the GAO report, as well as on the MSPAP scores. Yet that is all lost on you as you sit there pondering rainbows.
LikeLike
Chris likes to bring up MSPAP, but the scores are meaningless unless you have something to compare them to.
As any public school teacher in Maryland can tell you, the grading and scoring were modified, changed or adjusted many times during the first few years the test was used.
If Chris would like to use MSPAP, he needs to tell what was going on in the state of Maryland-ie, state averages, school district averages, etc.
Think of how Maryland has changed the criteria for a “Proficient”, a “Advanced” or a “Basic” score on the MSA, which is used for NCLB AYP determination.
He quotes the GAO, but the GAO had this to say of the CTBS scores:
Although small, the cumulated effects for CTBS
scores in both reading and math were statistically significant in favor of
the comparison group (ie, not EAI schools).
andThree of the seven privately managed schools (Dr. Rayner Browne, Harlem
Park Elementary, and Sarah M. Roach), however, show a more definite
pattern of underperformance compared with their matched schools. In
these cases, effect sizes were great enough to warrant further attention.
(thanks for the link to the eduwonk page)
The cut off has dropped. Don’t quote me, but I believe “proficient” is determined to be 50% correct on the exam.
LikeLike
Ed:
You’re going to try again with the creationist insults? Is that because I have more of a background in science than you do?
Again: does it make any difference whether her scores were from 16 years ago or rounded to 20? You’re a dimwit trying to make a big thing out of nothing, yet that’s the gimmick of this latest “scandal”.
And NO, I don’t “like to bring up MSPAP”. It was originally brought up by someone in another thread who had a knack for your type of debate (“Creationist! Rhee lied!”). He had said the MSPAP data showed no big gains. On the contrary, there is low performance on that test in 1993 and ’94, which leads into a positive shift of students scoring at satisfactory or higher in 1995, while this shift mostly fades away by 1996.
You are also ignoring my main point on that data and the rest: either you can use it to suggest there were large gains in 3rd grade in 1995 (although not separated at the teacher level), or you can’t. Either way, it doesn’t matter for my purpose in suggesting y’all need to learn how to read research papers, as it cannot be used to show/suggest that Rhee lied. At all. Not one bit.
Finally, try looking at the data cited in the GAO report, which shows gains at Harlem Park in 1995 that no other comparable school could match.
I would provide links, but they will be “moderated” into oblivion, as has already happened by your fellow truth-seeker Bradenburg. Readers can check out eduwonk . com for the extended debates.
LikeLike
You’re a dimwit”
No, Chris, you are.
As you know, if you want to put up a post with a link, drop the h t t p : / / part.
You are still providing false information on MSPAP as I noted above.
You are agnostic on the issue of the “acclaim” Michelle Rhee received from hartford Courant.
You like to pretend 20 and 16 are the same.
I thin k Kevin Johnson tried the same with the underage girls he was seeing.
The data on the CTBS scores have been broken down and show, no 90% at 90th percentile possible, atleast at the level that makes Miss Rhee’s claim true.
And even your idol is back-pedaling. Go read jay Mathews and Alecxander Russo. Maybe you can get a gig with Michelle as her data go to guy.
LikeLike
I wonder why Chris S feels so threatened? What is his personal interest in this?
LikeLike
She said that back in 1995, her report had been almost completely ignored: “it fell like a stone in the water at the time.”
That could be, but I thought the report, especially the bit about EAI spending more per pupil than BCPS, led to the city council canceling the contract with EAI in December 1995. (I wonder if EAI hit upon the use the public money to pay consultants scam.)
LikeLike
Ed:
Yes, that’s right, ignore what I write, as that comes naturally for you.
Look at the MSPAP scores. Note the sizable increase in proficiency levels during 1995 that isn’t there in years before or after. There is no way to look at that data and claim that Rhee lied. Look at it and say it suggests nothing, sure, but there is no “a-ha!” moment for your side of this debate from that data set. That was my point for your protege, Tedconsumer.
Similarly, look at the GAO report. Harlem park was the only school to improve on the reported CTSB scores, even up to a point that brought it in line with its comparison school. Can’t say that this shows she lied, either (although you sure tried your darndest). It also didn’t report scores broken down by teacher.
Finally, were Rhee to literally come out and retract the claim tomorrow it would NOT change the fact that with the given evidence, RIGHT NOW, you are wrong. This is bigger than just her scores– this is about valuing the possibility of reasoned and honest debate in education, an idea you continually disrespect with every nuance-averse comment you write. You both have publicly flaunted your ignorance of how to correctly read, and cite for support, research findings, and you continue to eschew all attempts to correct you, while fooling other people into thinking that what you’re doing is good for kids. It’s not.
And no, my “idol” (really?) has not apologized or retracted the claim. There’s no official data available to prove the 90th percentile claim (although the data, as flimsy as it is, still may suggest large gains), so it’d probably be better to not even mention it. However, there are two issues here:
1) Whether or not she should have made the claim despite a lack of officially verifiable evidence at hand,
and
2) Whether or not the evidence available support your claim that she lied.
For #1, considering the issues of transparency with the data at the time, it’s wrong to say that she’s without a doubt at fault. There’s nothing that says it was malicious, either: look at your own resumes and see if any awards or distinctions you have listed from 15-20 years ago can be 100% verified with papers you have with you at hand, without the need for a boss or work place to confirm them. It’s easy to consider a situation where she didn’t expect the data to be so hard to find, or didn’t expect any need by an employer to require more proof of a past achievement beyond a past boss’ confirmation, a past achievement that wouldn’t have any bearing on her ability to lead schools (and still doesn’t). Yes, she brought up the claim in public appearances as well, but never did she frame those scores as an indicator of her ability to lead public schools (and hopefully no one ever thought they would be such an indicator). The only thing she mentions them for is to show others what has most inspired her to continue the work that she’s done at TNTP, in DCPS, and now at StudentsFirst.
It is false to say that this all points to wrongdoing or confirms some other conspiracy theory you’d like to propose. You can keep your tinfoil hats on, sure, but spare me the proselytizing.
For #2, the issue is much clearer: there’s readily available data that you’ve poorly interpreted. I don’t know if that’s malice or ignorance on your part, but in either case it doesn’t fix the glaring faults in your analyses. It’s sad that people are so willing to buy what you’re selling, and so I don’t know what that says of the larger education community, but hopefully it’s something that can be fixed.
LikeLike
On what page, precisely, do you see the MSPAP scores for 1994-5 in that report?
LikeLike
Note the sizable increase in proficiency levels during 1995 that isn’t there in years before or after.
Since you are hung up on MSPAP, what were increases for the state that year?
What were the increases at Pimlico, the comparison school?
For #2, the issue is much clearer: there’s readily available data that you’ve poorly interpreted.
Funny, as you and I (independently) came up wit the same formula.
(# of Rhee*77+# of others*t)/(# of Rhee + # of others)= NCE
t= the NCE of the others
For reading it is NCE=45 and (# of Rhee + # of others)=56
# of Rhee=.9*# of total students she taught
# of total students she taught equals or is greater than 39 (19+20, the class sizes seen by UMBC researchers
For your numbers to work, the NCE for the others would be>1
Impossible
To get a more reasonable NCE, the number of Rhee students drops to lower than 21.
21 is not 90% of 70 that Miss Rhee claimed, nor 90% of 39.
The impact is that Miss Rhee is pushing an educational agenda that’s based on fantasy.
Stay on her plane, but I’ll catch you when you jump.
Guy, the MSPAP results are listed on an archive web page of the MSDE website
LikeLike
MSPAP Data here:
http://www.msp.msde.state.md.us/downloadindexprevious.aspx?WDATA=download
LikeLike
Over at eduwonk, Chris accepts Michelle Rhee at face value when she says she taught 70 kids. (2 classes of 35 each).
So, according to Chris, there are 63 1995 3rd graders who scored at the 90 percentile.
Where?
He and Hess argue that we don’t know that any of those UMBC listed scores belong to her students.
LikeLike
Chris and Rick are living in a statistical fantasy land. And, if none of those UMBC-reported scores belong to any of Rhee’s students, then she made up the ENTIRE brag from whole cloth. Which doesn’t make their argument any better!
LikeLike
@GF:
You brought up an interesting musing in an earlier comment: what is Chris’s motivation? This man has tirelessly posted seemingly dozens of lengthy posts here, on eduwonk, and elsewhere. His arguments have shifted from insisting Rhee could achieve the 90% on the test (something the study’s author confirms is impossible) to saying that other tests and reports say she did a good job. It’s borderline obsessive.
LikeLike
I have no idea what his motivations are, what he does for a living, or anything.
I seem to recall him saying he was a TFA teacher for a while. So maybe he drank a little bit too much of that cult’s Kool-Aid?
LikeLike
Guy, thank you for all you have done. The press is beginning to dig beneath the surface, thanks in large part to you.
LikeLike
Guy,thanks for doing the research and statistical work on Ms. Rhee’s less than miraculous work.
LikeLike
Chris is like the henchmen over in Tehran who are now attacking the pro-democracy demonstrators.
I played Chris’ game and tried to post his website.
My comment has been vaporized by Chris’ “moderation” of his blog(ue)
And he whined, claiming Guy refused to print his rants.
LikeLike
[…] unimportant. The evidence that she exaggerated her teaching prowess is, after all, inconclusive (though highly suggestive). A little resume inflation from a job 20 years ago might be overlooked, so long as Rhee’s […]
LikeLike
Thanks, Guy, for your good work. Here’s the latest and greatest fabrication yet:
http://www.schoolsmatter.info/2011/02/michelle-rhee-liar-who-just-keeps-on.html
Keep carrying the fire,
Jim
LikeLike
I came across a couple of interesting quotes from Miss Rhee’s principal, Linda Carter.
These appeared in the Washington Times back in 2007:
“Linda Carter, Harlem Park’s former principal, said after her testimony that she at one time had a document that listed Harlem Park test scores by grade level and school comparison. Those scores showed significant gains, Ms.Carter said, and she told the council that she had seen documentation showing gains of more than 50 percent in at least the third and fifth grades at Harlem Park.”
http://www.washingtontimes.com/news/2007/jul/3/rhee-vows-to-brighten-schools/print/
The there is this quote:
“Mrs. Rhee, who was in her early 20s while at the school, said she did not remember the size of her class.”
http://www.washingtontimes.com/news/2007/jun/28/schools-nominee-fails-to-validate-success/print/
That 5th grade teacher should be dug up and made the next DCPS chancellor.
LikeLike
Oh, that’s rich. She didn’t even remember how many students she had!!
“Gains of more than 50 percent” means precisely the same as when advertisers say “New and Improved and 50% better” . That is, of nothing.
Fifty percent of what?
LikeLike
Ed:
Again, MSPAP data was brought up by someone else who claimed they were evidence against Rhee’s students’ gains. I’m saying there is improvement in those data in 1995 that does not exist before or after, and that this trend also ties in with the other available data that suggests gains in that year for her grade level. I can look up Pimlico’s scores, too, but I’m not relying on MSPAP data to say anything other than those cut scores follow the trend found with the other data reported in the GAO and UMBC reports, and whether they on their own specifically say anything is debatable.
Regarding your math: you must realize that having the right formula doesn’t directly translate into a competent analysis. It’s the numbers you cite that are erroneous.
For instance, you reference the fact that 56 scores are tabulated for 2-year students in 3rd grade, but then make the assumption that the majority of those scores are Rhee’s students (“# students she taught equals or is greater than 39”). That is incorrect simply because Rhee does not reference all 70 students when she made the claim of 90% of them reaching such improvement. When she does reference all 70 students, it’s not to say that 90% of them made specific gains, either. She is most likely referring to the students she was the primary teacher for, a number that probably hovers at around 20 given the class sizes of the school.
It’s also not explicitly apparent why those 56 scores *must* have come from Rhee’s classroom and the classroom of her team teacher. Similarly, the 56 scores tabulated for 2-year students in the study doesn’t match the 70 students she’s referenced before. There are holes in the data here that you are sloppily filling yourself with your own understanding of the classroom breakdown, and it’s very flimsy argumentation.
The “educational agenda based on fantasy” line is also false since her agenda is not based on her classroom success 16 years ago.
Brandenburg:
I’ve already given my motivations in my comment above. You ought to consider them before you continue with this bizarre “Rhee-lover” business.
“And, if none of those UMBC-reported scores belong to any of Rhee’s students, then she made up the ENTIRE brag from whole cloth”
It’s obviously not all-or-nothing here. There are most likely scores from Rhee’s students reported in the UMBC study. There is uncertainty, however, in HOW many of Rhee’s students are a subset of the study’s reported numbers. That’s the sticky part of this whole debacle.
Josh:
“His arguments have shifted from insisting Rhee could achieve the 90% on the test (something the study’s author confirms is impossible)”
Don’t misquote me. I’ve not written much at all about insisting she is telling the truth. I’m much more interested in correcting the lot of you who are transparently bad at reading and citing the study’s findings.
I also couldn’t care less if you resent my “dozens of lengthy posts”. It’s a DEBATE, and it hardly implies obsession. Learn to accept dissenting opinion, particularly when you’re browsing the blogosphere.
Phillip:
Your comment was a playground jeer and a link to this blog. It was not a response to the debate, as I had given. I’d rather you stay on the swings if that’s what you want to contribute.
LikeLike
[…] of the scores of the remaining students had a modest increase. The lead analyst for the UMBC study agreed with the conclusions I drew. Rick Hess of the right-wing American Enterprise Institute, a personal friend of Rhee, predictably […]
LikeLike