Mathematical Hubris, or Simply an Author Who Doesn’t Understand Statistics?

My brother, who works in urban planning, called and told me I should read the article “X And The City” in the current issue of Smithsonian Magazine.

I did, and was quite disappointed. Here are my thoughts:

———————-

Hi, <brother>,

Thanks for pointing out to me the article on urban math in the current, May 2013 Smithsonian.

.

I was fully expecting to be quite enlightened and entertained, as I am by most Smithsonian articles, but I have to report that I was quite disappointed by it, and thought that the author was being naive. A lot of the conclusions seemed to me to be contrived or invalid.

.

I felt strongly enough to write this letter to compose my thoughts carefully.

.

The author does write, correctly, that “Cities are particular: You would never mistake a favela in Rio for downtown LA” and that many large cities will be surrounded by what they call ‘slums’ and others call shantytowns — developments put up informally and outside of any bureaucratic or official network of laws or public services of any sort at all, and which often exhibit a lot of negative behaviors and outcomes for their residents as a result.

Some of those bad effects are lack of public schools, no safe drinking water, no urban sanitation system, no safe and corruption-free police system, no public health facilities of any sort, no safe and reliable and dependable transportation system, no reliable electrical or postal delivery system, no zoning or building safety regulations that mean anything, and no real defense of the private property or land-ownership rights for those who have only small amounts of them. (Defense of private property only goes to the very rich and powerful. Matthew, you know.)

.

As the article correctly notes, we have very little dependable data on most of those shanty-town dwellers: neither how many people nor how much they earn or spend in the underground economy. A recent article I read indicates that cash payments are quite common in the US as well; so much of what is written about GDP per capita, anywhere in the world, is guesswork at best.

.

You know the saying about computer systems: Garbage IN, Garbage OUT.

.

Maybe I missed something, but I don’t see anything in this article that would allow any individual or group to use any of this data to do anything that would concretely help anyone in any significant manner.

.

One specific quibble concerns building heights. “…the equation H=134+0.5G where H is the height of the tallest building in meters and G is the Gross Regional Product in billions of dollars”… but previously, the author says the relationship isn’t strong. Well, how strong is it? It’s not clear at all. This page shows very different metrics, and rather different conclusions.  When I look up the so-called ‘Zipf Law’ I find that a number of people think it’s a trivial and unimportant correlation that one will find in almost any distribution of random-sized objects.

.

I did amuse myself by making this log-log graph with standings of population sizes of cities in the US.

This reminds me very much of graphs I used to see at the Naval Research Lab’s gamma-ray astronomy section, where they would have the logs of the energy of various gamma=rays that hit the Compton gamma-ray telescope on the left hand side, and the count of how many such photons on the bottom axis. Only the graph of the line of best fit went up to the right, not down to the right.

.

And those gamma rays were just about random, coming from anywhere in the universe. Weaker ones were much more common than strong ones.

A couple of minutes of work on a table from Wikipedia giving the masses of the largest solar system bodies and I get this graph:

Also apparently works for usage of words in ordinary language. Some are used very very often (like “the”, “is”, “of” and so on) and others hardly ever (“disestablishment”, “cornucopia”, “prolix”). You get a power law distribution. Not so special with cities, then.

.

================

I like the part where one of the people interviewed said “it’s just a coincidence” about correlation angle of sunrise and street numbers….

.

Only 50-60 years ago, New York City had the world’s tallest buildings, and had the record for a long time. Until a few city promoters/corporate idiots in Dubai and Kuala Lumpur and Hong Kong decided they would blow a few billions of dollars in building essentially useless tallest-in-the-world status symbols.

.

That being said, the data on heartbeats of and life expectancies of mammals do apparently fit a nice logarithmic line. That’s real data that anyone can measure — but now that I think about it, animal life expectancies very much depend on conditions, and critters living in zoos or labs are quite different from those in the wild… so I wonder how good even this data is… And are those resting heartbeats, or what?

.

In another case discussed in the article on making a decision whether to drive or take the subway to a Yankees or Mets game in NYC, or just to go home and watch it on TV if the traffic is bad enough as measured by Twitter or GPS on cell phones — I am skeptical, though I know that our smartphones have traffic-reading capabilities that do a fair, but not perfect job of showing you why you are stuck in traffic.  Seems to me that the decision on whether to go to a major-league sporting event is only partly based on traffic, and a lot on other value judgements that are not even considered.

.

Anybody who lets the folks in charge of technology decide for you how to spend an afternoon and evening needs to think again.

Mathematics of March Madness

It’s the season for the “March Madness” college basketball tournament, and it’s also the season when people place bets on exactly what the outcome will be – not the exact scores, if I understand correctly, but which teams will win each game. If you didn’t already know, MM is a single-elimination contest among 64 different excellent varsity college basketball teams; one loss and your team is out. If your team wins every single game it plays (six games), your team is the national champion.

Inspired by seeing an office pool and a recent book on how some gamblers ‘fix’ sporting events all over the world in order to win more money, I got to wondering how difficult it would be for someone like me who knows almost nothing about college basketball to predict the exact outcome of the tournament. Not just which team will takes the ultimate trophy, but which team will lose and which team will win, in all of the many individual matchups? (Forget predicting point spreads — you need a lot of inside knowledge to get that right, and that’s precisely the sort of thing that gamblers pay referees and players to cheat on.)

My question is simpler: exactly how many combinations and permutations will there be of who will be the winners of each and every single game of any year’s March Madness?

Thinking that tackling the actual problem would initially be too difficult, I decided to try a simpler question. What if there are only two teams? Obviously, only 2 outcomes: either team A wins, or team B wins, end of story.

How about if there are four teams? (Note: for there to be no “byes” and for this to be single-elimination, this column only considers cases where there are 2^n teams: 2, 4, 8, 16, 32, 64, and so on.) The answer was not obvious, so I drew a little diagram of all possible cases, which I roughly reproduce with MSPaint, here:

I called the teams A, B, C, and D. As you can see, there are exactly two ways that team A can win, two ways that team B can win, two ways for team C to be victorious, and two ways for team D to win – a grand total of 8 possible outcomes.

Another way of thinking of this is NOT to list and count all of the possible outcomes, but to figure out a way of counting WITHOUT counting.

I looked at the same situation and realized that there were two outcomes possible for each and every game, and you can multiply the number of possibilities at each node (location where two branches meet) branch by the number of possibilities at each other branch to get the same result, as you see here:

In other words, there are 2 x 2 x 2, or 2^3, or 8 possible outcomes, the same result I had before.

If there are eight teams, there are seven nodes, as you can see here:

Which works out to 2 x 2 x 2 x 2 x 2 x 2 x 2 outcomes, or 2^7. Notice that 7, the exponent, is one less than 8, the number of teams, just as 3 (the exponent in the previous case) was one less than 4, the number of teams. Does this pattern continue?

Yes, it does! I took my previous drawing and extended it to 16 teams, mostly by copying and pasting, which doubled the number of nodes, then I added one more at the very bottom. So this gave me 7 x 2 + 1, or 15 different nodes, or 2^15 different possible outcomes, and, as before, 15 is one less than 16.

Consequently, continuing the same pattern, if there were only 32 teams in March Madness, then there would be 2^31 different possible outcomes. In the current, real-world March Madness, there are 64 teams, so there must be 2^63 possible ways that the entire tournament could come out. (Which, parenthetically, is the number of grains of rice on the very last square of the mythical rice-doubling-award that was legendarily asked by the inventor of the game of chess…)

Is that a large number? Heck, yes! In fact, the numbers get very quickly very huge.

2^1 = 2

2^3 = 8

2^7 = 128  (over a hundred)

2^15 = 32,768  (over thirty thousand)

2^31 = 2,147,483,648 (over two billion)

2^63 = about 9,223,372,036,854,780,000 (my computer can’t exactly count that high, but it’s over 9 quintillion!)

If you were counting the seconds since the Big Bang, which apparently occurred over 14 Billion years ago, you would be nowhere near that result. You would in fact need roughly twenty lifetimes of the entire universe to count that number of seconds…

March Madness indeed. And, knowing as little as I do about which teams are better or worse, I would be indeed mad to try to bet on the outcome. What about you?

Details on those ‘Dozens and Dozens’ of Schools

As promised, I am posting the charts that I laboriously put together from data published by OSSE and a spreadsheet leaked to me, showing how unconvincing was the progress at the only 13 schools that WaPo reporter Jay Mathews could find that even vaguely resembled the ones that Michelle Rhee bragged about.

I am not going to look at the schools with large percentages of white students or at McKinley Tech.

First, here are my charts for Payne ES, Plummer ES and Prospect LC. I color-coded the chart much the way that Erich Martel does. That is, each diagonal sloping up to the right represents an individual cohort of students as they move from grade to grade, from year to year. Obviously I have no way of telling how many students transferred into our out of each cohort, but my experience in several DC public schools in all four quadrants indicates that students don’t move around all that much.

Thus, at Payne, under “Reading”, in the column for 3rd grade, in the row for 2010, you see the number 27. That means that at Payne, in the 3rd grade, in school year 2009-2010, about 27% of the students were ‘proficient’ or ‘advanced’ in reading according to their scores on the DC-CAS. The next year, most of those kids returned as fourth graders in the same school, and about 23% of them ‘passed’ the DC-CAS in reading because their answer sheets had enough correct answers for them to score ‘proficient’ or ‘advanced’. Note, that cell is also blue. But the next year, 2011-12, the percentage of students ‘passing’ the DC-CAS doubled, to 46%. I find that jump worthy of a red flag. Either that teacher did something astounding, or the students are an entirely different group of kids, or else someone cheated (most likely not the students).

 

Any time I saw a drop or rise of 10% or more from the year before for a single cohort, I noted my “red flag” by writing the  percentage of passing scores in bold, red.

Notice that at Plummer, the cohort in blue, in reading, went fomr 40% passing to 18% passing to 46% passing in the course of three years. The cohort in green in math went from 60% passing to 18% passing to  29% passing.

At Prospect, the cohort in yellow goes from 25% passing to 0% passing to 5% passing to 5% passing in reading. In math, the same group goes from 13% to 31% to 0% to 24% passing.

You see anything weird with that? I sure do.

Next come Thomson, Tubman, Hart, and Sousa:

 

The only one of these schools with a chart not covered with ‘red flags’ is Hart.

Your thoughts? (As usual, the “comment” button, below the end of this story, is tiny. Sorry about that.)

 

If you’re keeping score…

A handful of graphs and a bit of analysis of where are the highest and lowest-scoring students: in the regular public schools of Washington, DC, or in the publicly-financed but privately-run charter schools.

If you buy the current “party line” from most newspaper editorial boards and folks like Arne Duncan, Michael Bloomberg, the Koch Brothers, and Michelle Rhee, you would probably conclude that students in the charter schools are wildly outperforming students in the regular DC public schools.

Facts, as someone once wrote, are stubborn things.

It just ain’t so.

Look at these two graphs, which show bars that depict what percent of students in each of the public and charter schools are proficient in math:

The chart shown above is for all of the regular DC Public Schools. Notice that there are 15 schools (out of 117, or about 13% of the total number of schools) with proficiency rates over 80%.

Now let’s look at the graph for the DC charter schools:

Here, there are only four schools (out of 70 charter schools, or about 6%) that have 80% or more of their students scoring at what is called “proficient”.

What about reading? The situation is very similar. For the regular DC public schools, the chart follows here:

Here, there are 14 regular DC public schools out of 117 with student bodies where 80% or more of the students are “proficient” in reading on the DC-CAS. That’s 12% of the schools.

And in the charter schools, in reading, here is the graph for SY 2011-2012:

We see that there are only TWO (2) charter schools out of 70, or about 3%, where 80% or more of the students score “proficient”.

As I’ve written before, the regular DC public schools not only have the lion’s share of the high-flyers, so to speak. They also have the lion’s share of the low-achievers as well.

In math, there are 17 regular public schools, or about 15% of the schools, where less than 20% of the students are proficient in math. In the charter schools, there are only two schools (3%) with such low rates of proficiency.

In reading, there are 19 regular DC Public Schools (about 16%) with less than 20% of the student body proficient. In the charter schools, there are only two such schools (again, 3%).

By the way: none of this data is published at the regular NCLB/OSSE/DCPS data location, at least not yet. There are so far no breakdowns of student populations at each school by gender, race/ethnicity, proficiency in the English language, special education status, family income, AND grade — which is why I haven’t published anything on that. Seems to me that as time goes on, DCPS, charter schools, and OSSE are all releasing less and less information to the public.

I got this data here:

http://osse.dc.gov/release/mayor-vincent-c-gray-announces-2012-dc-cas-results

Consumer Math

Since my days as in HS and undergrad, computers have gone from something that was extremely rare but not impossible, all the way up to complete ubiquity in the US and much of the rest of the world (I don’t know exactly how far, but they have now reached tiny villages with no electricity or running water in Benin, west Africa, according to my niece who did a stint in the Peace Corps there: she tells me that lots of village and town folks had cell phones, which are, essentially, small, special-purpose computers. The relative cheapness of computers means that computational tasks that were only done by experts with access to rare computers, some years ago, can be done instantly and easily with Excel or an ordinary scientific or graphing calculator, which is on your smartphone anyway. (A safe bet that you, my reader, have one.)

I was one of the early folks with access to some sort of computer. In 1965, being a scholarship kid at a good New England boarding school, we had access to some early mainframe computer located at Dartmouth College (in NH like my school). We accessed the main computer via WATS lines (a fore-runner of the 800 phone system) and an honest-to-goodness modem that held the handset of the telephone in a cradle. We used some operating system referred to as “time-sharing” which meant that nobody at any of the 10 or 50 or 100 remote or local consoles noticed the presence on the computer of anybody else, which meant that we didn’t have to punch IBM cards and carefully arranged in trays and submit them to a computer operator — and then to await our turn in the queue in “batch mode” – the exact opposite of time sharing.

At that HS, we had a teletype printer that printed everything on a long roll of paper. We loved the thing and even imitated its type font. I was scornful of those kids who wrote simulations of baseball or football games, thinking that wasn’t serious. Boy, was I wrong! The games my schoolmates were writing in our spare time at the computer lab, after sports and classes and before bedtime, fit in our schedule around the time needed for completing our other homework assignments, were incredibly crude by our standards today, of course (the computer would describe the flow of the game in a few words and then ask you if you wanted to throw a fastball, a curve, or a sinker or a knuckleball, and then the computer would work out a pseudo-random number and make a decision as to what happens next (grounder to first, double play, pop fly, safe hit, double, home run, etc). But the genre of computer games has obviously earned its makers and promoters a TON of money over the years and have attracted the very, very best programmers and computer artists of all types. So don’t take everything I say as the gospel truth. Nor the words of anybody else, even your very own self.

(/brag & reminiscence ON/ I won a third place contest when I wrote a program that would do tons and tons of calculations and then print out, on the same roll of teletype paper, a very crude graph of absolutely any equation the user wanted, at any scale that might be desired, using an algorithm that I devised as a modifrication of some spaghetti code I had seen somebody else write. I mean even equations that were as complicated as

Sin(x^4.7*sqrt(tan(y))-log(x^2-y^9)+(e^cos(5x-2y^3))=0.

/brag & reminiscence OFF/)

I remember my brother and lots of sociology or psychology students having to make computer runs in various semi=specialized statistics packages that calculated all sorts of stuff about all sorts of data. (Usually the computer’s output was that you had made some sort of syntax or usage error and you needed to fix the error and try again later, whenever that might be.) I remember collecting IBM punch cards and computer printouts from the recycling bins at the University of Maryland so I could use them for scratch paper and flash cards in my classroom, back in the late 1970s and early 1980s. I also remember having to go to UM or UMUC or GWU or GtnU or CUA or AU or UDC in order to do various programming assignments in various languages . I, personally, only punched a handful of  Fortran cards, treating it as more of a historical curiosity that i was glad I never had to use, similar to the way that folks look at slide rules today.

But there is a difference: I understood exactly how every part of the punch card mechanisms and its electronic alter-egos worked together. I could write the Fortran code (not well, but I took part of a course and taught myself from books, before going on to learn many other computer programming languages. However, I bet that most folks, even those who have just finished a year or semester of working with logarithms, could not explain how or why a slide rule worked.

Let me name a few of the computer languages I have learned, more or less in order, and in no particular order as to how well I learned them:

BASIC (in lots and lots of different variations), COBOL, FORTRAN, Logo (lots of varieties), Pascal, C-64 and Apple 2 assembly and machine language, IDL, Python. Then I said that was enough.

I also learned how to use probably a dozen different types of typewriters and later word processors, and also was quite serious about calligraphy for several decades. Old Underwood standard manual typewriters were quite different from portable or electric typewriters.

Naturally, I also used lots of different types of computational aids, Let’s try listing them:

• fingers
• abaci
• pencil-and-paper (learning and modifying various shortcuts along the way),
• mental arithmetic (shortcuts, estimation)
• and an entire panoply of calculators, ranging from the $400 calculator in 1977 dollars that could only do X, /, +, – and not even square root, on up to the ones that we can now program to do all sorts of things, including somewhat basic graphics. And lots of different spreadsheets (I just found some wonderful spiffy videos on how to do all sorts of different tasks using the latest version of Excel. Some of them really do make things much, much easier. But getting accustomed to this new version is, again, a learning process, once again.

I am sure that I am not the only person who thinks that it is not necessary to upgrade all electronic stuff all he time. Sure, the newest versions generally crash less often than the old ones, run faster, and have lots of new features, hut it’s rather expensive to keep having to buy new stuff. It especially doesn’t pay to be an early adopter, especially if you choose wrongly and you end up owning devices that are abandoned by the market and all of your hard-won detailed operating knowledge becomes useless.

Wait. That happens with everything these days!

Including cars. Here’s a little story:

The Subaru Forester my wife and I own was starting to cost a serious amount of money to repair after 10 years and 100,000 miles. We got an estimate from our most dependable mechanic that needed repairs to the window, trim, and transmission would be close to three grand on top of several grand in other repairs over the past year. Plus it would still remain noisy and not have good gas mileage, and the head gasket might be leaking, too.

When a car’s blue book value is not very much greater than the estimated repair bill, my solution is to start looking around for another car.

Not to fix it myself any more. These big jobs were never in my capacity to fix, even back when working on a car was pretty easy. In the old days (1920s through the mid-1980s) it was kind of essential for a guy to understand how to fix stuff on the car, and it often wasn’t too complicated — just really dirty and greasy. (The vast majority of women wouldn’t bother, as you probably remember, which gave rise to lots of jokes.) But today, the situation is different: almost nobody has the equipment, time, and necessary training to work on their own car. If you look under the hood, there isn’t any room left, and it’s extremely complicated to boot. Just to diagnose many problems you need a special-purpose computer.

I remember around 1980 that sometimes you could just disassemble an inoperable part such as a starter motor, clean it out really well, put it back together again, perhaps replace a washer or a nut, then reinstall it, and it would work quite well for some years. Plus, in some older Big 3 cars, one or two smallish people could easily bend over and fit in the engine compartment with the hood closed. So almost everything was easy to reach and take off and re-install.

But today, with modern cars, and going back to at least electronic ignition, a/c, and serious emission controls, all the way up to futuristic cars like the one we just bought (a prius v, level 3), you can’t do any of that.

Yeah, we bought a Prius V, which looks like it’s almost in a Matrix body, and we came to this conclusion based on some simple but useful consumer math. Our old car, the one that just reached 100 thousand miles, was great for many purposes. Perfect when it snows (except we had NONE this winter), fits my very large home-made telescopes and camping gear for remote dark-sky locations, and can hold many other things as well. It worked well up to 100 thosand miles, but then it started falling apart as I described. Our daughter had an old Subaru Forester like we did, and it also fell apart, but after more miles. (anything involving headgaskets is quite expensive!) BUT it had a big downside or two.

The overall gas mileage was around 20 mpg, which I think sucks. (Nearly every time we filled the tank over the years we would estimate what the mileagte (i.e. mpg rating) was. Some times it was a little bit below 20 mpg, sometimes a bit more. 20mpgt as an overall average is close enough and easy to work with. es, there are plenty of bigger cars that guzzle more fuel, but a Forester isn’t all that big. AWD is nice, but it really uses a lot of gas. I was envious of those who drove hybrids, but wasn’t sure they made sense for me, financially and so on.

Plus, Subarus are really noisy. You can’t stop the air from making wooshing sounds hecause of the way they made the windows, at least afrter the fist year or so. Plus, there ae lots o little rattly things all over the underside of the car – my mechanic’s approach was to take them off, one by one.

Now, Feb/March 2012, it began to make sense to buy a Prius, and let me explain why.

At 20 miles per gallon, 100,000 miles consumed roughly 5,000 gallons of fuel. It looks like gasoline is inching up to a long-term average of about $4 per gallon, which is what it used to cost in Europe about 10 years ago. So, using $4/gallon as a rough guide, that five thousand gallons costs $20,000. If I bought another forester, it would probably cost that much to fuel it in the next 10 years, all other things being equal.

However, the larger Prius V is supposed to get 41 to 45 mpg according to the famous EPA estimate. I used 40 mpg to be conservative on this for various reasons. If we drove a Prius 100,000 miles in 10 years and it gets 40 mpg, then it will use 2,500 gallons of gas. And at the same price of $4/gallon, that gas would only clost $10,000, which is half as much.

I don’t know about you, but to me, ten thousand dollars is a lot of money. It made me want to take a look. If I can save ten grand on a new type of car that can do the same thing but better than my old type of car, then that calculated $10K savings on fuel makes a big difference. It meant that the roughly $10K difference in price between a Forester and a Prius V could be essentially discounted.

Why not buy a used car? When we were young and broke (beginning teachers back in the 1970s and 1980s didn’t earn squat) my wife and I had a series of used cars, sometimes gifts from relatives (as we did for our own kids later on) and sometimes from private sellers or dealers. Some of the cars were great, some were absolutely horrible. So horrible that we laugh out loud because retelling the stories is so deliciously funny in a weird sort of way. (You could see the road through the floor! There were wasps in the upholstery! No dipstick, jack, no bald tires, trunk full of leaves!) But it’s really a huge hassle to sell your old car through classifieds or auto magazines or craigslist or whatever, and then to get a new one.

So a new car, traded in, a hybrid Prius V it is.

It’s like driving a computer. (Perhaps you’ve seen this before; I’ve driven in one a few times, but I find this a revelation.) The gas gauge almost never seems to move (exaggeration!). If I want it to, I can have the car tell me at every instant which way the power is flowing, including from the wheels through the electric motor back to the battery when I’m braking. The car’s display often claims that I’m getting over 100 mpg for a 5 minute stretch, especially when I’m coasting downhill or coming to a stop at a light or intersection. I don’t have to use the key in either the ignition nor the door, and I can tell my iphone to call my wife or anybody else. It claims its calculated the mileage at 40.3 mpg overall, but I can’t yet tell if that’s accurate, because I’m still at a half a tank of gas, so I can’t yet do the little elapsed miles divided by gallons purchased estimate to see what my real mpg is since when we bought it. We will see in a week or so.

Math is, truly, everywhere, and is part of our lives.

We all use it; better to use it well, without too much effort, than to use it poorly or only with great effort. I think we could do a much better job in school of showing how useful it really is. By that, I don’t necessarily mean we should teach formal algebra courses in 5^th grade. I mean what I wrote, show kids lots of ways that they really do need to use math in many ways that are not in the old-fashioned curriculum more or less put in place in 1893 or thereabouts.

And since computers are, literally, all around us, let us use them in wise and intelligent ways that save us a huge amount of effort, to answer questions that mean a lot to us. Yes, you show kids how to do peprform standard algorithms or certain variations thereof. But you also show them how to use calculators and spreadsheets and other graphic or mathematical electronic tools as well.

A Page For Folks Who Might Find that they Like Math

A web page with very fun explorations that show what real mathematicians do:

http://mathlesstraveled.com/2012/02/09/17×17-4-coloring-with-no-monochromatic-rectangles/

(Hint: long division, balancing checkbooks, and adding fractions aren’t such a big part. I suspect that if we taught some of this stuff in class, kids would see that, in fact, math has some fun aspects — it’s not all drudgery. There is a lot of “gee-whiz” stuff that is accessible to the average layperson or young’un.

The Rest of the 2011 Mathematics Baccalaureat exam

As promised, here is the last section (four pages, two long exercises, each one made of subsections and subquestions) of one of the French Mathematics Baccalaureate exams. This version is one for folks who did serious secondary work in mathematics and science. This was a lot of very slow, technical typing, and trying to figure out whether I want to give a literal translation, or whether I should reach for easier understanding by American readers. I tried to opt for the second choice.

Be prepared: this section of the test is rough going, too. If you can’t quite even figure out what they are asking, you are in very good company! Only folks who have been specifically and recently studying or teaching these particular concepts would have a chance of passing.  Even with a score of 45%.

(page 4:)

and page 5:

page 6:

 and finally, page 7:

If you’ve read until here, you might think that the math part of the regular SAT is almost trivial in comparison to this test I translated. But remember this: kids raised in the French tradition and who attend French schools, find that a lot of the content of the American SATs is a mystery to them as well.

Real Data on NAEP scores in DC over the past 20 years

I would like to remind you that Michelle Rhee’s and Richard Whitmire’s claims that she single-handedly boosted the NAEP scores in DCPS, way in excess of anything that happened previously, are bogus.

Michelle Rhee and Richard Whitmire aren’t left with very much to brag about concerning her supposed accomplishments in DCPS.

• The ‘Capital Gains’ project was a total failure – even according to the data provided by Harvard professor/entrepreneur Roland Fryer (who has also run a massive randomized experiment in NYC showing that paying teachers money for higher student test scores was counterproductive)
• Bringing in an outside agency to run Dunbar SHS was a total failure
• The situation with the least-trained, least-experienced teachers staffing most of the roughest, lowest-income schools in DCPS is worse now than ever
• The situation with special education is the usual mess
• There are more central-office employees than ever, and they make more money than ever
• It’s clear that the enormous sums she spent on consultants haven’t done a damned thing
• The teachers are totally demoralized
• Real teaching has been replaced by 100% test-prep, all the time
• The method that her favorite principals employed to raise test scores involved simple cheating: long hours of meticulous erasing of wrong answers, after the students and teachers had left the buildings

So let’s look at a few graphs of how NAEP scores have changed over the past 20 or so years, so you can see for yourself whether the 3 years of Rhee made much of a difference.

First, please look at a graph of the average 4th grade DCPS ‘Scale Scores’ on the Math NAEP:

As you can see, they have been rising more-or-less steadily since about 1996.

Next, look at a graph showing the percentage of students “Basic” or above (which really does mean ON GRADE LEVEL)  and “Proficient” or above (which really does mean ABOVE GRADE LEVEL) in math since about 1992. Again, you can see that these percentages have been pretty steadily rising since about 1996. Once again, SuperWoman’s presence apparently didn’t make a huge change.

Next, let’s look at the average scale score for DCPS 8th graders on the NAEP reading test:

Here the record doesn’t go back as far, but besides the drop in 2003 and 2005, the overall trend has been upwards.

I’ll post a few more graphs tomorrow. Right now, I am too tired to keep going.

DCPS Administrators Won’t or Can’t Give a DCPS Teacher the IMPACT Value-Added Algorithm

Does this sound familiar?

A veteran DCPS math teacher at Hardy MS has been asking DCPS top administrator Jason Kamras for details, in writing, on exactly how the “Value-Added” portion of IMPACT teacher evaluation system is calculated for teachers. To date, she has still not received an answer.

How the “Value-Added” portion of the IMPACT actually works is rather important: for a lot of teachers, it’s about half of their annual score. The general outline of the VAM is explained in the IMPACT documents, but none of the details. Supposedly, all of the scores of all of a teachers’ students’ in April are compared with all of those same students’ scores last April; and then, the socio-economic status and current achievement scores of those students are taken into account somehow, and the teacher is labeled with a single number that supposedly shows how his or her students gained during that year with respect to all other similar students.

But how those comparisons are made is totally unclear. So far  I have heard that the algorithm, or mathematical procedure, that has been used is designed to make it so that exactly half of all teachers are deemed, in non-technical terms, ‘below average’ in that regard — which of course will set them up to be fired sooner or later. Whether that’s an accurate description of the algorithm, I don’t know. Ms. Bax told me that she heard that DCPS expects that teachers with almost all Below-Basic students would be expected to achieve tremendous gains with their students. However, my own educational research indicates the opposite.

In any case, Kamras and his other staff haven’t put any details in writing. Yet.

At one place Kamras writes that “we use a regression equation to determine this score.” OK, Bax and Kamras and I all teach or taught math. We all understand a fair amount about regression equations. But there are lots of such equations! Just saying that there is a regression equation is involved is like saying Newton “used algebraic equations” in writing “Principia Mathematica”, or that Tolstoy used “words and sentences” when he wrote “War and Peace.” And just about equally informative.

I attach a series of emails between Ms. Bax, an 8th grade math teacher, and Mr. Kamras and a few other people in DCPS Central Administration. The emails were supplied to me, in frustration, by Ms. Bax. I used color to try to make it clear who was writing what: Green is Ms. Bax, and reds and browns and pinks denote those written by for various administrators. Note that this exchange of emails started in September of 2010.

Perhaps publicizing this exchange might prod Mr. Kamras to reveal details on a system that has already shown by Mathematica (the same group that designed the system) to be highly flawed and unreliable?

=========================================

From: “Bax, Sarah (MS)” <sarah.bax@dc.gov> Date: Mon, 13 Sep 2010 17:12:41 -0400

To: Jason Kamras jason.kamras@dc.gov Subject: Impact

Jason,

I hope the year is off to a great start for you.

I am writing concerning the IMPACT IVA score calculations.  I am very  frustrated with this process on a number of fronts.  First, I would like to have an actual explanation of how the growth scores are calculated. As they have been explained, the process seems quite flawed in actually measuring teacher effectiveness.  Further, I would like to know if teachers have any recourse in having their scores reexamined, etc.

Last year, 89% of the eighth graders at Hardy scored in the Proficient or Advanced range in Mathematics.  As the sole eighth grade mathematics teacher last year, I taught almost all of the students except for a handful that were pulled for special education services.  Beyond this accomplishment, I am extremely proud to report that 89% of our Black students were at that Proficient or Advanced level.

With statistics like these, I take issue with a report that scores my IVA at 3.4 (to add insult to this injury, even under your system if my students had earned just one-tenth more of a growth point, my IVA would be a 3.5 and I would be considered highly effective).

Frankly, I teach among the best of the best in DCPS– with very few of us rated highly effective.  The IMPACT scoring system has had a terrific negative impact on morale at our school.

Kindly,

Sarah Bax

———————————————–

From: Kamras, Jason (MS) Sent: Tue 9/14/2010 7:50 AM To: Bax, Sarah (MS) Subject: Re: Impact

Hi Sarah,

I’m disappointed to hear how frustrated you are. Can you give me a call at 202-321-1248 to discuss?

Thanks,

Jason

Jason Kamras

Director, Teacher Human Capital

——————————————————–

Jason,

I really do not have the time to call to discuss my concerns.  If you would forward the requested information regarding specific explanation about the growth scores calculation process I would be most obliged.

I would like specifics about the equation.  Please forward my inquiry to one of your technical experts so that he or she may email me with additional information about the mathematical model.

Kindly,

Sarah

—————————————————

From: Barber, Yolanda (OOC) Sent: Mon 12/20/2010 2:12 PM To: Bax, Sarah (MS)

Subject: FW: IMPACT Question

Ms. Bax,

Sorry for the barrage of emails, but I received a response concerning your question.  Please read the response below.  I hope this helps.  Please let me know if you’d like to continue with our session on the 4th.  Thanks again.

Best!

Yolanda Barber

Master Educator | Secondary Mathematics

District of Columbia Public Schools

Office of the Chancellor

——————————————————–

From: Rodberg, Simon (OOC) Sent: Monday, December 20, 2010 2:05 PM

To: Barber, Yolanda (OOC); Lindy, Benjamin (DCPS); Gregory, Anna (OOC) Subject: RE: IMPACT Question

Hi Yolanda,

We will be doing more training, including full information on Ms. Bax’s question, this spring. We’d like to give a coherent, full explanation at that time rather than give piecemeal  answers to questions in the meantime.

Thanks, and I hope you enjoy your break.

Simon

Simon Rodberg

Manager, IMPACT Design, Office of Human Capital

———————–

Yolanda,

I got notice a couple of weeks ago that I have jury duty on your office hours day at Hardy so I won’t be able to make the appointment.  I’m sorry to miss you, but appreciate your efforts to send my concerns to the appropriate office.

The response below is obviously no help at all as it clearly indicates the Office of Human Capital is unwilling to answer my specific question regarding the calculations involved in determining my rating.  I believe my only request was to have an accurate description of how the expected growth score is calculated.  My question has been left unanswered since last spring.  Can you imagine if a student of mine asked how his or her grade was determined and I told them I couldn’t provide a coherent explanation right now, but see me in a year?

Thanks again for your help.  I look forward to meeting you in person in the future!

Kindly,

Sarah

——————————-

From: Bax, Sarah (MS) Sent: Tuesday, December 21, 2010 11:09 AM To: Rodberg, Simon (OOC)

Cc: Henderson, Kaya (OOC) Subject: FW: Appointment #80 (from DCPS Master Educator Office Hours Signup)

Mr. Rodberg,

I am requesting a response to my inquiry below:    ‘explanation of actual algorithm to determine predicted growth score’.

Kindly,

S. Bax

—————————-

Ms. Bax,

What’s a good phone number to reach you on? I think it would be easiest to explain over the phone.

Thank you, and happy holidays.

Simon

Simon Rodberg

Manager, IMPACT Design, Office of Human Capital

—————————–

From: Kamras, Jason (DCPS) [mailto:jason.kamras@dc.gov] Sent: Sun 12/26/2010 1:42 PM

To: Bax, Sarah (MS) Cc: Henderson, Kaya (OOC) Subject: Value-added calculation

Hi Sarah,

The Chancellor informed me that you’re looking for a more detailed explanation of how your “predicted” score is calculated. In short, we use a regression equation to determine this score. If you’d like to know more about the specifics of the equation, please let me know and I can set up a time for your to meet with our technical experts.

Happy New Year!

Jason

Jason Kamras

Chief, Office of Human Capital

—————————————

On 12/27/10 12:17 PM, “Bax, Sarah (DCPS-MS)” <sarah.bax@dc.gov> wrote:

Jason,

I have requested an explanation of the value-added calculation since September, with my initial request beginning with you (see email exchange pasted below).  I would like specifics about the equation.  Please forward my inquiry to one of your technical experts so that he or she may email me with additional information about the mathematical model.

Kindly,

Sarah

————————————

On 12/27/10 12:23 PM, “Kamras, Jason (DCPS)” <jason.kamras@dc.gov> wrote:

My deepest apologies, Sarah. I’ll set this up as soon as I get back.

Jason Kamras

Chief, Office of Human Capital

—–Original Message—–

From: Kamras, Jason (DCPS) [mailto:jason.kamras@dc.gov] Sent: Tue 1/25/2011 11:02 PM

To: Bax, Sarah (MS) Subject: FW: Value-added calculation

Hi Sarah,

I just wanted to follow up on this. When could we get together to go over the equation?

Hope you’re well,

Jason

Jason Kamras

Chief, Office of Human Capital

———————————-

From: Bax, Sarah (MS) Sent: Fri 1/28/2011 1:15 PM To: Kamras, Jason (DCPS)

Subject: RE: Value-added calculation

Jason,

I really would just like something in writing that I can go over– and then I could contact you if I have questions.  It is difficult to carve out meeting time in my schedule.

Kindly,

Sarah

————————–

From: “Bax, Sarah (DCPS-MS)” <sarah.bax@dc.gov> Date: Thu, 10 Feb 2011 14:05:43 -0500

To: Jason Kamras <jason.kamras@dc.gov> Subject: FW: Value-added calculation

Jason,

I didn’t hear back from you after this last email.

Kindly,

Sarah

———-

From: Kamras, Jason (DCPS) [mailto:jason.kamras@dc.gov] Sent: Thu 2/10/2011 6:00 PM

To: Bax, Sarah (MS) Subject: Re: Value-added calculation

Ugh. So sorry, Sarah. The only thing we have in writing is the technical report, which is being finalized. It should be available on our website this spring. Of course, let me know if you’d like to meet before then.

Best,

Jason

Jason Kamras

Chief, Office of Human Capital

————————————-

On Feb 25, 2011, at 9:29 PM, “Bax, Sarah (MS)” <sarah.bax@dc.gov> wrote:

Jason,

How do you justify evaluating people by a measure [for] which you are unable to provide explanation?

-Sarah

————–

Sat, February 26, 2011 11:25:33 AM

Sarah,

To be clear, we can certainly explain how the value-added calculation works. However, you’ve asked for a level of detail that is best explained by our technical partner, Mathematica Policy Research. When I offered you the opportunity to sit down with them, you declined.

As I have also noted previously, the detail you seek will be available in the formal Technical Report, which is being finalized and will be posted to our website in May. I very much look forward to the release, as I think you’ll be pleased by the thoughtfulness and statistical rigor that have guided our work in this area.

Finally, let me add that our model has been vetted and approved by a Technical Advisory Board of leading academics from around the country. We take this work very seriously, which is why we have subjected it to such extensive technical scrutiny.

Best,
Jason

Jason Kamras
Chief, Office of Human Capital
——————————-

Jason,

To be clear, I did not decline the opportunity to speak with your technical partner.  On December 27th I wrote to you, “I would like specifics about the equation.  Please forward my inquiry to one of your technical experts so that he or she may email me with additional information about the mathematical model.” I never received a response to this request.

In addition, both you and Mr. Rodberg offered to provide information about the equation to me on the phone or in person, but have yet to agree to send any information in writing.  You have stated, “I just wanted to follow up on this.
When could we get together to go over the equation?”  Mr. Rodberg wrote, “What’s a good phone number to reach you on? I think it would be easiest to explain over the phone.”

Why not transpose the explanation you would offer verbally to an email?  Please send in writing the information that you do know about how the predicted growth score is calculated.  For instance, I would expect you are familiar with what variables are considered and which data sources are used to determine their value.  Let me know what you would tell me if I were to meet with you.

As a former teacher, you must realize the difficulty in arranging actual face-time meetings given my teaching duties.  And as a former mathematics teacher, I would imagine you could identify with my desire to have an understanding of the quantitative components of my evaluation.

Sincerely,
Sarah

Staff Development in DCPS

It has been stated that under Michelle Rhee, funding for staff development in DCPS quadrupled.

I suspect that is possible that this might be true in the very narrowest sense, but only if you pick the comparison times very, very carefully.

During Michelle Rhee’s first two years in DCPS, there was almost no real staff development worth the name.

I recently retired (June 2009) after over 30 years of teaching math in DC public schools. (Among other things, I coached a number of DC JHS and MS math teams to state championships, usually beating many or all of the other MathCounts teams from all other DC public, private, parochial, and charter schools.) I endured two full years of Michelle Rhee’s misguided leadership before leaving the classroom.

In the two-year period August 2007 through June 2009, I can honestly say that we had the very fewest system-wide content-specific or methods-specific staff development sessions that I can ever recall in my three decades in DCPS. At least that was the case in mathematics. Years ago, I recall attending numerous staff development sessions on teaching math, and on other topics, at my local school and at various DCPS or outside buildings, run either by DCPS teachers and staff, or else by outside experts. (I won’t say they were all wonderful, because some were not. But some were in fact great, and some of those good ones were run by other DCPS teachers.) I ran a few of such session, myself, sometimes for free, sometimes for a small stipend. (I hope that the sessions I ran were useful and not too boring, but will let others judge their quality.)

The only such PD session that I recall DCPS running in my subject (math) from fall 2007 through spring 2009, was introductory sessions on using graphing calculators.

Sorry, but that is a joke.

No person should be placed into a classroom and be asked to teach math as a subject if they have never used a graphing calculator before and needs such a course. If there are one or two such people in the system due to some unforeseen disaster, then they should be personally summoned for a one- or two-hour session on the topic, or someone should go to them and show them. By contrast, just about every mathematics teacher I know in DCPS could in fact TEACH either an introductory, an intermediate, or an advanced class or series of classes on using graphing calculators.

However, a couple of months ago, someone showed me where one could find the recent DCPS budgets for special purchases. I soon discovered that during SY 2009-2010, many millions of dollars have been spent under Rhee and Kamras on consultants for staff development. From what I understand, almost all of these sessions are run by relatively inexperienced, so-called ‘master teachers’ (often graduates of “Teach For Awhile”) who tell the rest of the teachers what they must do in order to measure up under the IMPACT measurement scheme. And, it appears, there are lots of such sessions. What’s more, judging by the figures I saw being paid to TFA and NTP and NLNS, these consultants are being remunerated at very, very generous levels.

So, four times what? Compared to when? And, if you simply pay the same consultants four times as much for the same work, is that really even germane?

Does this current type of staff development actually work to improve teaching?

Or, as George W. Bush might have said, “Is the children learning more good?”

Time will tell.

It is certainly instructive that after all this hugely increased emphasis on teaching to IMPACT during the last full school year, DC-CAS scores in DCPS abruptly plummeted after rising for several straight years.It might be obvious from the tenor of my comments that from what I hear, the current funds for staff development are mainly being used to browbeat teachers, not really to improve teaching.

And, what’s more, it would be better if teachers had:

* some say in what sort of staff development sessions they could attend, and

* had more freedom to craft their own good lessons.

Guy Brandenburg, Washington, DC

__,_._,___