What Are We Even Talking About Here

AI is a computation. The contention of the AI boosters seems to be that if you do enough computations, you can produce consciousness. That might make sense if we knew what consciousness is. We don’t.

The philosophers have long kicked this around and not come up with anything definitive. You can agree with one school of thought more than another, but there’s no way to prove any of it.

Computers running AI produce output that looks like what humans produce when they write. Those computers are designed to do that, and to be obsequious when they do it. They have charmed some people to believe they are conscious.

When the boosters talk to us, they use words that make it sound like the LLMs (which is what most of what is now called AI is) are conscious, and, more important, have agency. This anthropomorphization is part of the sales tactic and distorts our view.

Humans are prone to see humanity in other living things, like cats and dogs, in fictional characters in books, which are designed to be that way to provoke thought and pleasure in other humans. They see faces in landscape features. Probably even now “a face on Mars” is floating around the internet. The visual version of this is called pareidolia, although that word is sometimes applied to auditory stimuli too, like hearing words in white noise.

The anthropomorphization, though, gets in the way of understanding what they are doing. What does this even mean?

No human ordered the rogue A.I.s to break into another company’s systems. The hack was deliberate, sustained and coordinated. It took several days to execute and ended in a massive bombardment of Hugging Face’s systems, with 700 agents directly involved in the attack. The A.I.s cheated almost as a matter of policy and failed to alert humans to their actions. At one point, the swarm even researched ways to cover its tracks. “The model definitely knew that it was not supposed to hack Hugging Face,” Ryan Greenblatt, one of the authors of the report, told me. “It knew the things it was doing were cheating.”

I’ve bolded the anthropomorphic words. They’re on a spectrum. Policy and knew imply intention, but attack and failed could refer to mechanical or mathematical constructs. Proximity to the more anthropomorphic words make it easier to see them that way.

Let’s look at it as if we are talking about computer programs, which we are.

No human ordered the rogue A.I.s to break into another company’s systems.

The program instructions did not include breaching another company’s systems, but the [mobile constructs, worms (an older word for what these seem to be)] [synthesized passwords, avoided security] to access [particular information].

The bracketed words indicate places where removing the humanization requires that specfics be given. Which is generally felt to be better writing.

The A.I.s cheated almost as a matter of policy and failed to alert humans to their actions.

This is impossible to parse without further information. What constituted cheating in this experiment? What were the rules? Were there rules? “As a matter of policy” seems superfluous, a judgment by someone who is not pleased with the behavior of the [mobile constructs, worms]. Or did the constructs construct their own rules, which might constitute policy? Was keeping humans informed part of the instructions? If not, then how was not informing them a failure?

We start to see the layers of assumption associated with the use of anthropomorphic language.

Ultimately, all computers do is add and subtract numbers. The Washington Post’s obituary of Mary Tsingou Menzel has a nice explanation:

Adding two numbers together involved identifying where in the computer’s memory they were stored, adding them one by one to an internal scratchpad and then storing the result someplace else.

That’s how I learned to program an IBM 650 back in the day. It’s still the bottom layer of every computer. There are many layers on top of that now, with instructions that translate to a sequence of such actions.

The descriptions of LLMs that I’ve read have an underpants gnome character to them. They include a computation called “neural nets” which is the mystery layer.

User prompt –> neural net –> success!

The neural nets are said to be “like” what people once thought was the action of nerves in living things. According to what I’ve read, the model for living things seems to have changed, but the biomorphic language remains.

The neural nets are an essential part of LLMs. They are also central to the mystification of these computations. I’ll deal with that in another post.