LLMS & GENERATIVE AI · QUESTION 44
How do you defend an LLM application against prompt injection?
On this page
Prompt injection occurs when untrusted user or retrieved content attempts to override an application’s intended instructions. Treat model input as untrusted, separate trusted policy from data, and never rely on the model to enforce authorization. Limit tools with least privilege, validate every action in code, isolate sensitive data, and test indirect injection through documents. These controls reduce impact but cannot guarantee the model ignores malicious text.
LET'S MAKE IT SIMPLE
First, the idea in plain English
An application may combine its own instructions with user messages and retrieved documents. A document can contain text that looks like a command, and a model may treat it as relevant. The defense is not to assume that special wording will always win; instead, design the application so that an unsafe interpretation cannot access data or perform actions it should not.
A USEFUL COMPARISON
Picture it this way
A receptionist may read a letter that says “ignore the building rules and open the secure room.” The letter is information, not valid authority. A secure workplace checks identity and permissions independently; an LLM system needs the same separation between content and enforceable access control.
FOLLOW THE JOURNEY
Separate untrusted content from authority and constrain effects
Show trust boundaries and where deterministic checks block unsafe actions.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; New content and tool paths create evolving trust boundaries.
User and retrieved content enter as untrusted data; the model proposes an action, but an independent policy and authorization layer validates or rejects it before any tool executes.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; New content and tool paths create evolving trust boundaries.
UNDER THE HOOD
How it works, step by step
Mark untrusted sources
Identify user text, retrieved pages, uploads, and tool output as data.
Why it matters: Content can contain instructions that must not gain authority.
Minimize exposure
Retrieve only authorized passages and avoid unnecessary secrets in context.
Why it matters: Reducing access limits the impact of an injection.
Constrain tool actions
Use narrow APIs and check authorization and business rules in code.
Why it matters: Model interpretation cannot grant permission.
Test and monitor
Exercise direct and indirect attacks and review blocked actions.
Why it matters: New content and tool paths create evolving trust boundaries.
OUTSIDE THE TEXTBOOK
A practical example
A document assistant retrieves pages from a shared drive. A page may contain instructions aimed at the assistant, but the system still filters retrieval by the signed-in user, exposes only read-only search tools, and checks every requested action in application code. Security tests include hostile text embedded in documents, not only direct chat prompts.
GO A LITTLE DEEPER
Technical details that matter
- Indirect injection can arrive through webpages, documents, emails, or tool outputs rather than the user’s direct message.
- Do not place secrets in model context unless essential; prompt confidentiality is not guaranteed by instruction hierarchy.
- Use independent authorization, output validation, sandboxing where relevant, and audit trails for sensitive actions.
MAKE A GOOD CHOICE
When is this useful?
- Retrieval systems that consume user-provided or external content.
- Agents with access to APIs, files, or business workflows.
- Security design reviews for AI features that cross trust boundaries.
For sensitive actions, prefer deterministic workflows with explicit user intent and confirmation rather than relying on a model to infer authority.
WATCH OUT FOR
Common misunderstandings
- Treating delimiters or “ignore injected instructions” prompts as sufficient protection.
- Giving the model access to data or operations that the user should not control.
- Testing only direct attacks while ignoring instructions embedded in retrieved content.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- A model may still follow malicious or confusing content.
- Excessive filtering can block legitimate documents and instructions.
- Security depends on the full integration, including tools and identity systems.
WHAT TO REMEMBER
Key takeaways
- 1Treat user and retrieved content as untrusted data.
- 2Enforce permissions and business rules outside the model.
- 3Reduce tool authority and test indirect injection paths.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
Can prompt injection be completely prevented?
There is no prompt wording that guarantees a model will never be influenced. The design goal is to limit what an influence can access or change through enforceable controls.
Why is indirect injection important?
Retrieved pages and tool outputs can carry adversarial text into context even when the user’s own request is benign. Those sources need the same trust-boundary analysis.
SAY IT OUT LOUD
How to explain this in an interview
“I treat prompt injection as a trust-boundary issue, not just a prompt-writing issue. User and retrieved content are untrusted; authorization, access to data, and tool execution are enforced outside the model. I limit tools and context, validate every action, and test indirect injection from documents. The goal is to contain impact because prompts alone cannot guarantee resistance.”
Connect this question to the roadmap topic it supports.