LLMS & GENERATIVE AI · QUESTION 01
What is a Large Language Model (LLM), and how does it work?
On this page
A large language model is a neural network trained on large collections of text to predict likely next tokens. When given a prompt, it converts the text into tokens, uses a Transformer to process their context, and generates a response one token at a time. It learns patterns rather than checking every statement against a trusted source, so its answers can sound plausible and still be wrong.
LET'S MAKE IT SIMPLE
First, the idea in plain English
An LLM is a program that has learned patterns in language from examples. Give it a prompt and it estimates what piece of text is likely to come next, then repeats that process to form a response. Text is split into tokens—small units such as words or word fragments. A Transformer uses attention to weigh how parts of the input relate to one another. The model’s fluency comes from learned patterns, not a built-in guarantee that each sentence is true.
A USEFUL COMPARISON
Picture it this way
Imagine a conversation partner who has read an enormous library and is very good at continuing sentences in a fitting way. They can explain patterns they have seen, but unlike a careful librarian they may not open a source to verify a claim. The comparison helps explain pattern-based generation; a model does not remember or understand as a person does.
FOLLOW THE JOURNEY
From a prompt to a generated response
See how input becomes model-readable tokens and is turned into a response incrementally.
Each generated token is fed back as context for the next prediction. The exact serving pipeline can include additional safety and formatting steps.
A prompt is tokenized, processed by a Transformer using context, and used to predict and append tokens until a response is complete.
How to read this visual
Each generated token is fed back as context for the next prediction. The exact serving pipeline can include additional safety and formatting steps.
UNDER THE HOOD
How it works, step by step
Prepare the text
A tokenizer maps the prompt into token IDs.
Why it matters: The neural network operates on numbers, not raw characters.
Represent context
The Transformer updates token representations using attention across the available context.
Why it matters: Words can mean different things depending on what came before.
Predict a continuation
The model assigns scores to possible next tokens; a decoding strategy chooses one.
Why it matters: This is the basic generation objective learned during training.
Repeat and stop
The chosen token joins the context and the model predicts again until a stopping rule is met.
Why it matters: One-token-at-a-time generation creates the final sequence.
OUTSIDE THE TEXTBOOK
A practical example
Ask a writing assistant to explain why leaves change color. The prompt is tokenized, the model uses surrounding words to predict a relevant continuation, and the application displays the generated tokens as a sentence. If the prompt asks for a very specific local fact, fluent generation alone does not ensure the answer is current or correct.
GO A LITTLE DEEPER
Technical details that matter
- During pretraining, the model adjusts learned parameters to reduce the error of predicting tokens in training sequences. The training text is not stored as a simple searchable database.
- Transformer attention lets each position combine information from other positions that are visible to it. In a decoder-only model, a causal mask prevents a position from using future tokens.
- Temperature and other decoding choices affect how the next-token distribution is used. They influence variation, not factual reliability.
- A context window limits how much input and generated text can be considered at one time.
MAKE A GOOD CHOICE
When is this useful?
- Drafting, summarizing, and transforming language.
- Conversational interfaces when the application can check or constrain important outputs.
- Code assistance paired with review and execution in a safe environment.
For exact facts, use authoritative retrieval or structured data; for deterministic rules, use ordinary software. A language model may still help explain those results.
WATCH OUT FOR
Common misunderstandings
- Treating fluent output as proof of truth.
- Assuming the model searches the web whenever it answers.
- Describing next-token prediction as a guarantee of human-like understanding.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- Training data can be incomplete or out of date.
- Ambiguous prompts and unfamiliar inputs can lead to confident errors.
- Long contexts and generation have latency and compute costs.
WHAT TO REMEMBER
Key takeaways
- 1An LLM generates text by predicting tokens from context.
- 2Transformers use attention to process relationships in the context.
- 3The model learns patterns; it does not automatically verify claims.
- 4Grounding, evaluation, and human review matter for consequential use.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
What is a token?
A token is a unit produced by a tokenizer, often a word, part of a word, or punctuation. Token counts affect context length and model cost.
What does attention do?
Attention lets a token representation combine information from other relevant positions in the input, rather than treating every word independently.
Why can an LLM hallucinate?
Its training objective rewards likely continuations, not guaranteed factual verification. Missing context or weak grounding can therefore produce plausible but incorrect text.
SAY IT OUT LOUD
How to explain this in an interview
“A large language model is a neural network trained to predict the next token in text. It first turns a prompt into tokens, then a Transformer uses attention to build context-aware representations. The model predicts a token, adds it to the sequence, and repeats to create a response. This makes it useful for language tasks, but it does not automatically verify facts, so important answers need grounding or checking.”
Connect this question to the roadmap topic it supports.