The LLM Style

Eve Fairbanks, whom I can attest is an extremely skilled editor, has some things to say about LLM-generated prose.

The problem is that the efficiency and frictionlessness that make AI appealing to writers are the same qualities that make it feel untrustworthy to readers. And readers are right not to trust it. No matter how much we may tell ourselves that AI is just a tool like spell-check, it isn’t. When we use AI to flesh out ideas, we lose the most important part of the writing process: thinking.

What’s wrong with the LLM style?

So we end up with canned perfection—writing that can’t really be argued with, because it has no underlying deliberative reasoning process, no train of thought. As I wrote on X recently, AI writing is almost impossible to edit, because even when it sounds plausible, a closer look will show that every element is equally off: The tone is bland; individual word choices are baffling; the structure lacks sense; key pieces of the argument are missing; facts are false. Working on AI text, as an editor, is like trying to operate on a body whose skin, muscles, veins, bones, and organs are all compromised. There’s nothing to leave intact, nowhere to begin.

Fairbanks illustrates this with an interaction that she had with Chat-GPT:

Recently I asked ChatGPT Pro—which I paid for to summarize financial spreadsheets; I’m not a hater—whether it was capable of viciously dismantling an argument I’d read about the stock market. “I’m tuned to avoid being casually cruel, contemptuous, or needlessly snide,” it explained. “That doesn’t mean I can’t be sharp, skeptical, funny, or sarcastic … I can absolutely be dismissive of bad arguments, flabby prose, obvious spin, bureaucratic nonsense, or market delusions wearing a little tech-futurist hat.”

I wanted to look under the hood a little, to understand the reasoning behind how the program chose its phrases. “Why would market delusions be wearing a ‘little’ tech-futurist hat, rather than a big one?” I queried.

The program was incapable of pausing to wonder whether it had chosen the wrong word. Instead, it sought to justify itself in increasingly bewildering ways. “Because ‘little’ makes it funnier,” ChatGPT Pro answered. “The market delusion wants to stride into the room as Prometheus; ‘little hat’ makes it look like a raccoon that found a conference badge.”

Why a raccoon with a conference badge? I pressed. Why not a dog, or a frog?

“Raccoons live in alleys, storm drains, garbage nights, and morally humid environments,” it wrote. “Frogs have existential agency, while raccoons have logistical agency. A frog at a VC afterparty seems like an omen. A raccoon at a VC afterparty seems like it got on the guest list … Think: alley behind a conference hotel, the loading dock of a casino, the VC afterparty where everyone is saying ‘alignment’ but meaning ‘exit liquidity’ … Alignment sounds like ethics, safety, shared goals, humanity, the future. It has dry-cleaned blazer energy … Exit liquidity is socially clammy.”

All of these sentences are grammatically perfect. They also make no sense. And all substantially AI-generated writing is like this, under the hood. ChatGPT

I share Fairbanks’ frustrations with LLM-generated prose, but I’m not sure about all of this. LLMs have no intentionality; they are predictive algorithms that produce, in the technical sense of the term, bullshit. Fairbanks asked the LLM to predict what text to provide in response to her query. Then she asked the LLM to predict what text to provide in response to her query about that text. My sense is that the exercise works well enough. I am less convinced that AI writing looks uncanny to someone who doesn’t spend a lot of time working with, let alone editing, text.

Regardless, I don’t have any big “takeaways” here. I just think it’s an interesting piece. It looks like it’s ungated. I’m curious what you all think about LLM-generated text in general, and her claims in particular.