Home > Uncategorized > Simpson’s Paradox and “Failing Schools”

Simpson’s Paradox and “Failing Schools”

May 7, 2017

Cathy O’Neill, aka the Mathbabe, writes extensively on the misuse and abuse of statistics. In a column that appeared in Bloomberg News, she described how a statistical phenomenon called “Simpson’s Paradox” might have resulted in some over-estimates of misogyny at Facebook. What is Simpson’s Paradox?

Simpson’s paradox, one of the trickiest and most intriguing obstacles to human understanding of statistics… holds that aggregate data can generate a completely different picture than subsets of the same data — but only if something interesting is going on in those subsets.

In the Facebook case where junior engineers are criticized more frequently than senior engineers and women are disproportionately underrepresented in senior engineering positions, it turns out that women are criticized more often than men… but it may have nothing to do with gender and everything to do with the job assignments.

But it was in describing instances where Simpson’s Paradox came into play in previous analyses that public education was impacted:

Blindness to Simpson’s paradox can be consequential. Back in 1983, the Nation At Risk Report, which spurred the modern education reform movement, presented the troubling finding that SAT scores were going down over time. What it didn’t say is that if you subdivided SAT takers by their high-school class rank, you’d find something completely different: In each subgroup, scores were either holding steady or improving.

The explanation was that more kids from lower class ranks — kids who tended to be from lower-income or minority families — were aiming for college, and therefore taking the SAT in the first place. Because they typically got lower scores, they brought the average down even though their own scores were improving over time. So actually pretty good news, but it wasn’t reported until years after the original report came out.

Well it WAS reported repeatedly in education journals like Phi Delta Kappa… but the “failing schools” meme played into the notion that anything associated with the government is a problem and so it was THAT message that drowned out reality. And so… nearly 35 years later after years and years of hearing about “failing “government schools”” we have reached a time when that is a given and vouchers are given credibility.

%d bloggers like this: