Junk numbers, general innumeracy, and the death of shame

Andrew Gelman picks up on what is apparently a quasi-official US government claim that every boat being sunk off Venezuela is saving 25,000 lives, and cites one of Bill James’s aphorisms about treating numbers as if they were words:

Bill James once wrote that his innovation as a sports analyst was to think of baseball statistics as numbers that could be added, subtracted, multiplied, and divided, in contrast to the usual attitude in which statistics are treated like words (so-and-so hit .300 or led the league in stolen bases or whatever).

We often see this meaningless-numbers-as-words attitude coming from credentialed academic social scientists. Some examples we’ve discussed over the years include:
– The claim that beautiful parents are 36% more likely to have girl babies (thanks, Freakonomics!),
– The claims that single women were 20% more likely to support Barack Obama and three times more likely to wear red or pink clothing during certain times of the month (thanks, Psychological Science!),
– The claim that every execution prevents 18 murders (thanks, Harvard!).

These are examples of what one might call standard-issue innumeracy, which is how we might characterize claims that could in theory be correct but whose plausibility disintegrates after any serious engagement with reality. These are numbers that don’t make a lot of sense but they kinda sound good. A moment’s reflection would cause immediate skepticism, but who has time for a moment’s reflection? Not Steven Levitt, Cass Sunstein, or various authors, reviewers, and editors for Psychological Science. The numbers don’t mean anything, they’re just a way to tell a story.

Standard-issue innumeracy can come by fishing in small samples of noisy data, yielding what can be massive overestimates of effect size.

Hard-core innumeracy

But then there are what we might call hardcore innumeracy, those quantitative statements that don’t even require a moment’s reflection to recognize as absolutely ridiculous. For example:
– The claim that the probability of a decisive vote is 10^-90 (thanks, British Journal of Politics and International Relations!),
– The claim that scientific citations are worth $100,000 each (thanks, Ted talks!).

As Campos might say, these are the equivalent of saying that a baseball player is hitting 3.000 or that somebody is on track to hit 272 home runs in April.

But credentialed social scientists write these things! What’s the point? 10^-90 is a really tiny number and $100,000 is a really big number, that’s the point.

This reminds me that people who study these things sometimes argue that phrases like “ten years” in Homer or “40 years” in the Bible just meant “a long time” in those texts, as opposed to something more literal. So maybe Trump is just following the literary stylings of God or Homer, and means “a lot of lives,” rather than some specific actual number, like the kind used by those pointy-headed statisticians and social scientists who think they’re so smart but could never reconstruct a ’69 Chevy with a 396, fuelie heads, and a Hurst on the floor.

In any event, Trump’s claims are actually quite modest in comparison to those of thot leaders like Pam Bondi:

One hundred days into Donald Trump’s second presidency, his Attorney General Pam Bondi, which is to say the most important law enforcement figure in the federal government, tweeted out a celebratory message regarding her Leader’s remarkable accomplishments.

“Today is Fentanyl Awareness day,” she wrote. “In President Trump’s first 100 days we’ve seized over 22 million fentanyl laced pills, saving over 119 Million lives.”  The next day, after some media criticized the absurdity of this number, Bondi raised the figure to 258 million lives saved, at a White House press conference called to respond to criticism of the original figure.

This claim had at least three purposes.

First, in regard to the distressingly large percentage of the public that is incapable of the most rudimentary statistical reasoning, Bondi intended that her message be taken as a literal statement of fact.  Consider that a 2022 poll of more than 2,000 Americans found  that those surveyed made the most extraordinary mistakes regarding such facts as how many people in the country had household incomes of over one million dollars per year, or were transgender, or Muslim, or Jewish.  The correct answers ranged from much less than one percent to up to two percent for these various categories, but per this poll Americans believe that the respective prevalence in the population ranged from a low of 20% (millionaire annual income) to a high of 30% (Jewish population).

It is, I suppose, not too incredible to imagine that people who make such estimates would not be particularly skeptical about the claim that Donald Trump’s fentanyl interdiction policies saved the lives of more than one third – subsequently raised to three quarters — of the nation’s population over the course of his first 100 days in office.

Second, the claim was no doubt intended to enrage opponents of the Trump administration – to “own the libs” in the jargon of our fascist jokesters.  Pam Bondi is a figure of almost pure malevolence, but surely she is intelligent enough to realize that her claims have no conceivable relation to, if we may use an old-fashioned term in these postmodern times, the truth (On the other hand I find it quite believable that Trump himself would not find Bondi’s claims obviously false, assuming he had any interest in their accuracy, which we can be confident he does not).

The claim, heard so often from his supporters during the first Trump presidency, that his statements were supposed to be taken “seriously, not literally” is just a variation on the classic fascist strategy of kidding on the square, meaning joking but with serious intent (fascist buffoonery is funny until someone gets hurt).

Third, and most crucially, the core purpose of Bondi’s outrageous lie is to undermine the very concept of some sort of reliable public discourse, especially one emanating from purportedly authoritative government sources.  A passage from Hannah Arendt’s The Origins of Totalitarianism captures this dynamic well in the context of the Nazi and Stalinist regimes:

In an ever-changing, incomprehensible world the masses had reached the point where they would, at the same time, believe everything and nothing, think that everything was possible and that nothing was true. … Mass propaganda discovered that its audience was ready at all times to believe the worst, no matter how absurd, and did not particularly object to being deceived because it held every statement to be a lie anyhow. The totalitarian mass leaders based their propaganda on the correct psychological assumption that, under such conditions, one could make people believe the most fantastic statements one day, and trust that if the next day they were given irrefutable proof of their falsehood, they would take refuge in cynicism; instead of deserting the leaders who had lied to them, they would protest that they had known all along that the statement was a lie and would admire the leaders for their superior tactical cleverness.

This kind of cynical willingness to both utter and swallow whole the most outrageous falsehoods was satirized in 1984, but that satire was, in the context of the Nazis and the Stalinists, uncomfortably close to a literal description of their tactics and psychology. 

From The Triumph of Stupidity (forthcoming eventually).

Innumeracy plays a big role here of course, but so does just ignorance in general. I happen to know with some precision how many people died in the United States in each of the last five years, and what the causes of those deaths were, because I have a freakish interest in such matters, but even somebody with only a very rough idea of the actual numbers would recognize how totally absurd Bondi’s and Trump’s claims are.

But as the percentages cited in the quote above illustrate, “only a very rough idea of what the actual numbers are” is not a level of knowledge that vast tracts of the common clay of the West have achieved.

One of Gelman’s commenters (Peter Dorman) adds what I think is a key point that helps explain what’s happening both here and in so many other social contexts:

I know I’m repeating myself, but I think the disappearance (or dwindling) of shame is an important part of the story. Once upon a time, shame was a real social force. If an author/politician/public figure made a claim and was shown to be wrong, they experienced shame and apologized. Or they fought back and tried to show there was nothing to be ashamed of, or they were devious from the get go to avoid later events that might be shameful. (Or they had arguments, trotted out in exceptional circumstances, that circumstances were exceptional and justified a bit of fudging.)

Over time, for probably a number of reasons, the shame effect went away. People can be shown to be in egregious error, or even just making stuff up, and they go right on. The norms surrounding public discourse have changed. It’s not that there aren’t people who dedicate themselves to examining the validity of the claims being made (thanks Andrew!), but that pointing out error or even meaninglessness has few consequences.

I think you can describe a corollary to this about innumeracy, as a particular type of BS claim tailored to appeal to other innumerates. There was a time when people with limited quant skills shied away from putting out specific numbers because they were afraid of being shamed by people who had the ability to debunk them. Not any more. And that goes for other areas of specialized knowledge. Look at the way JD Vance makes claims about US history, for instance. He doesn’t worry that someone who knows a lot more about this history than he does will disprove him.

It’s scary to think that LLM’s are being plopped down into this ecosystem.

I’ve seen this particular dynamic play out over and over again in recent years, both inside academia and well beyond it. And yes, throwing AI slop into this cultural mix is indeed terrifying.