What is an LLM?

The setup

The wrong mental model costs you hours.

Most people, asked what an LLM is, land on "an AI," or "a really smart search engine," or "I don't know, but it's amazing." None of those is right, and each one quietly costs you the same way: prompts that don't work, answers you trust that turn out to be wrong, weird behaviors that look like glitches but aren't.

Most AI training skips this. It starts with prompts — "ten prompts to make you 10x more productive." That works for a week or two. Then the prompts stop landing, the model seems to "get worse," and you assume the technology slipped. It didn't. You ran out of cases your borrowed mental model could handle, and there was no foundation underneath it to extend from.

A carpenter with a wrong model of how wood moves with humidity builds a beautiful cabinet that splits in winter. He didn't lack a tool. He lacked the underlying frame. AI is the same. The frame goes first.

"Most people who say 'ChatGPT got worse' actually mean their mental model ran out of cases. The technology didn't change. Their luck did."

The answer

What an LLM actually is.

An LLM is a prediction machine. You give it some text. It predicts what chunk of text most likely comes next, writes that down, then takes everything so far and predicts the next chunk. And the next. Until it stops. That's it. That's the whole thing.

It is not thinking. It is not reasoning. It is not "looking things up." It generates the next most-probable piece of text given everything before it. The fact that this produces something that sounds like thinking is a measure of how good the predictions are — not evidence that thinking is happening underneath.

The most honest shorthand is the one engineers use: auto-complete on steroids. Your phone guesses "soon" after you type "I'll be home" because it has seen the pattern. An LLM is the same idea, blown up by about six orders of magnitude — trained not on your texts but on a huge chunk of the public internet, books, and code; guessing the next chunk not from your last three words but from the last several thousand. Better data, more context, more compute. Same core operation: predict the next chunk.

One thing it did not do: learn facts the way you learn them. It never read a textbook, verified a source, or updated a belief when the world changed. It saw an enormous amount of text and got frighteningly good at predicting text that looks like it. That single fact is the seed of everything weird about how it behaves.

What the model predicts

Five "glitches" that aren't glitches.

Once you accept "prediction machine, not thinking machine," every behavior people complain about stops looking like a bug and starts looking like the obvious consequence.

And a fifth: it's instantly good at hard things and weirdly bad at easy ones. It will write a passable sonnet in three seconds and botch 517 × 388, because it doesn't do math — it pattern-matches what math answers tend to look like. None of these are flaws a future version fully fixes. They're consequences of the core design: predict the next chunk. Even when the next big model ships, this frame still holds.

The trap

What an LLM isn't — and what each wrong model costs you.

A short list of mental models people operate on, and the specific bill each one runs up:

• Not a search engine. A search engine retrieves documents that exist; an LLM generates text that sounds like such a document. People who think it's "looking it up" trust it like Google — then cite a fake statistic in a meeting.
• Not a person. No continuous identity, no memory of you, no real curiosity. It sounds like a person because it was trained on what people wrote. People who forget this over-share and overweight its "opinion."
• Not an oracle. For predictions, novel judgment calls, anything needing real-world experience, it pattern-matches similar text. That's not wisdom. People delegate decisions to a confident-sounding generator and find out after the fact.
• Not a database. Even a correct fact is a pattern-match, not a lookup. Two near-identical prompts can give different answers. There's no canonical version of a fact stored inside it to retrieve.

The cost is always the same shape: you trust the prose more than the prose has earned. The prose's relationship to reality is your responsibility, not the model's.

Why this matters

Four things that change once the frame clicks.

This is where the mental model pays for itself. Four operating principles fall straight out of "prediction machine, not thinking machine":

• Better input → better output. The model generates text conditioned on what you gave it. Vague input, vague prediction. The single biggest gain most people ever get is moving from one-line prompts to four-sentence prompts — same model, same task, far better answers.
• Specificity beats length. A long, rambling prompt can be worse than a short precise one. The model isn't impressed by word count; it's conditioned on information. One clear constraint and a bit of context beats two paragraphs of throat-clearing.
• Verify the load-bearing facts. Before you act, ask: if this is wrong, what does it cost? "Nothing, it's a draft email" — ship it. "I'd lose a customer or misquote a regulation" — verify externally first. Always. It's the only correct way to use a system that can't tell you when it's guessing.
• Treat output as a draft, not a deliverable. The first answer is rarely the best one. Read it, push back, ask for the opposite, ask what's missing. The best users have conversations, not one-shot transactions. "Type prompt, copy answer, paste into work" uses maybe 20% of the tool.

Where to take this next

The free curriculum picks up here.

Related reading on the site

If this landed, the next pieces.