Tokens, transformers, context windows, why LLMs hallucinate, and how to choose between Claude, GPT, and open-source models — explained in plain English.
Large language models power the AI tools transforming software, yet how they actually work is often shrouded in either intimidating mathematics or hand-waving mystique. The truth sits in between and is genuinely understandable without equations. Grasping the core ideas — what these models really do, how they were built, and why they behave as they do — makes you far more effective at using them, because you understand their nature rather than treating them as inscrutable oracles. This is how large language models work, explained plainly.
At their core, large language models do something surprisingly simple to state: they predict the next piece of text given what came before. Trained on enormous amounts of text, a model learns the patterns of language so well that, given some input, it can predict what word is likely to come next, then the next, and so on, generating coherent text one piece at a time. Everything these models do — answering, writing, summarizing — emerges from this next-word prediction performed extremely well. Understanding that the fundamental operation is prediction, not comprehension in a human sense, is the single most clarifying insight about how they work.
Models do not actually work with words exactly, but with tokens — chunks of text that might be whole words, parts of words, or punctuation. Text is broken into these tokens, and the model predicts tokens rather than words. This detail matters in practice because it affects how models process text and how usage is measured and billed, typically per token. Understanding that the model's unit is the token, a piece of text not always identical to a word, explains some of the model's behavior and is important for practical concerns like cost, since you pay by the token. It is a small technical reality with real practical implications worth knowing.
These models are built by training on vast quantities of text — much of the written internet and more — during which the model adjusts itself to get better and better at predicting the next token across all that text. Through this process over an enormous amount of data, it learns the patterns, structures, facts, and styles present in the text. The result is a model that has absorbed a great deal of what was in its training data. Understanding that the model's capabilities come from learning to predict text across a massive corpus — that its knowledge and abilities are patterns extracted from training data — explains both where its impressive abilities come from and where its limitations originate.
An important nuance is that the model learns patterns in language, not a database of facts. It does not store and look up facts the way a database does; rather, it has learned statistical patterns so rich that it often produces correct information, because correct information was common in its training. But this also means it can produce plausible-sounding but incorrect information, because it is generating likely text, not retrieving verified facts. Understanding this distinction — patterns versus a fact store — is crucial, because it explains why models are often right but sometimes confidently wrong, which directly shapes how you should and should not rely on them.
The tendency of models to sometimes generate false information, often called hallucination, follows directly from how they work. Because they generate plausible text based on patterns rather than retrieving verified facts, they will sometimes produce something that sounds right and fits the pattern but is simply not true. The model is not lying or malfunctioning; it is doing exactly what it does — producing likely text — in a case where the likely text happens to be wrong. Understanding that hallucination is an inherent consequence of next-token prediction, not a bug to be fully eliminated, is essential to using these models responsibly, with verification where correctness matters.
A model considers a limited amount of text at once when generating — this is its context window. Everything you provide and everything generated so far, up to that limit, is what the model uses to predict what comes next. It does not remember beyond a conversation's context or learn from your interactions; it works from what is in the current context. Understanding the context window explains both how the model uses what you give it and its boundaries — why it cannot recall things outside the current context and why providing relevant information in the context is how you make it work with specific data. It is a practical concept that shapes how you interact with the model.
A natural question is whether these models truly understand, and the honest answer is nuanced. They do not understand in the human sense — they have no consciousness, beliefs, or genuine comprehension. Yet they exhibit behaviors that look remarkably like understanding, because they have learned the patterns of language and reasoning so thoroughly. The practical stance is to treat them as extraordinarily capable pattern-based text generators that often behave as if they understand, while remembering they do not actually know things. Holding this view keeps you from both underestimating their usefulness and overestimating their reliability, which is exactly the balanced perspective that makes you effective with them.
Because a model's knowledge comes from its training data, it knows nothing about events or information after its training was completed — its knowledge has a cutoff. Ask about something recent, and the model either does not know or may fabricate. This is why providing current information in the context is how you work with up-to-date data, and why models are paired with retrieval systems for current knowledge. Understanding the knowledge cutoff explains a key limitation and its solution: the model's built-in knowledge is frozen at training time, so for anything current or specific, you supply it in the context rather than expecting the model to already know it.
Knowing that the model predicts text based on what came before explains why prompts matter so much. The text you provide sets the context that shapes what the model predicts next, so a clear, well-structured prompt that establishes the right context leads to better output, while a vague one leaves the model to fill in ambiguity. You are, in effect, steering the prediction by what you put before it. Understanding that prompting works by shaping the context the model predicts from — not by issuing commands to a thinking entity — demystifies why prompt quality matters and makes you better at writing prompts that reliably get the results you want.
This understanding translates into practical wisdom. Because models predict plausible text from patterns, verify their output where correctness matters and do not trust them for guaranteed facts. Because they have a knowledge cutoff and context window, provide current and specific information in the context. Because prompts shape prediction, write clear prompts. Because they generate rather than retrieve, expect variability. Each practical guideline for working with models follows from how they actually work. Understanding the mechanism is not academic — it directly informs how you build reliable AI features, which is precisely why grasping how these models work, even without the math, makes you genuinely more effective with them.
When a model predicts the next token, it does not pick a single certain answer but works from probabilities across possible next tokens, and there is usually some randomness in which is chosen. This is why a model can give different responses to the same prompt — the generation is not perfectly deterministic. Settings can make it more or less random, more focused or more varied. Understanding that generation involves probabilistic choice among likely continuations explains the variability you observe, and why you can tune toward consistency or creativity. It also reinforces that the model is producing likely text rather than computing one fixed correct output, which is central to its nature.
Beyond the initial training on vast text, models are often further shaped to be more helpful, follow instructions, and behave safely — a refinement on top of the base capability. This is why modern models are good at following your instructions rather than just continuing text, and why they tend to respond helpfully. You do not need to do this yourself; the models you use through an API have already been refined this way. Understanding that the models you interact with have been specialized to be helpful and instruction-following — not just raw text predictors — explains their cooperative, useful behavior and why prompting them with clear instructions works as well as it does.
One striking aspect of these models is that capabilities like translation, summarization, and answering questions were not explicitly programmed — they emerged from learning to predict text across an enormous, varied corpus that included all of these things. Because the training text contained examples of every kind of language task, the model learned to perform them. This emergence from general training is why a single model can do so many different things. Understanding that the model's broad abilities arise from comprehensive training rather than from being built task by task explains both its remarkable generality and why its behavior can sometimes surprise even those who built it.
Pulling it together, a practical mental model is this: a large language model is an extraordinarily capable text predictor that has absorbed the patterns of language and knowledge from massive training, generates plausible continuations probabilistically, works only from its training plus the context you give it, and has been refined to be helpful. It is not a thinking being and not a fact database, but a powerful pattern-based generator. Holding this mental model lets you anticipate the model's behavior — its strengths, its variability, its tendency to fabricate, its dependence on context — which is exactly what makes you effective at building with it, turning an opaque tool into one whose nature you genuinely understand.
Large language models, demystified, do one thing at their core: predict the next piece of text — the next token — given what came before, having learned the patterns of language from training on enormous amounts of text. Everything they do emerges from this prediction done extremely well. They work with tokens rather than exact words, which matters for cost; they learn statistical patterns rather than storing facts, which is why they are often right but sometimes confidently wrong; and that pattern-based generation is exactly why they hallucinate plausible falsehoods. They consider only a limited context window and have a knowledge cutoff at training time, so you supply current and specific information in the context. They do not understand in the human sense but behave remarkably as if they do. Crucially, prompts work by shaping the context the model predicts from, which is why prompt quality matters so much. Grasping these ideas — without any mathematics — turns the model from an inscrutable oracle into a tool whose behavior you can anticipate, which is what makes you genuinely effective at building reliable features with it.