What Is Context Engineering? A Guide for Business AI Applications

Prompt engineering asks how to phrase an instruction. Context engineering asks a wider question: what should the model be looking at when it answers? In business applications, it decides most of the result. A support assistant that sees the right order record and the right policy page answers well with a plain prompt. One that sees the whole knowledge base and every customer's data answers worse, costs more and creates a data protection risk.

Context engineering improves business AI applications by giving each request a small, relevant and permission-checked set of information, because models answer less reliably as their input grows and can only use the facts they are actually shown.

This article explains the parts of the context, how to budget it, how to keep it permission-aware and how four delivered projects changed what the model receives. It does not cover model training or fine-tuning, and it focuses on business applications rather than coding agents.

Key takeaways

• Context is everything the model receives for one request: instructions, the user's input, retrieved knowledge, records, tool definitions and results, history and output format.
• Language models do not use long inputs evenly. Research across 18 models found performance becomes less reliable as input length grows, even on simple tasks.
• Give each request a context budget, and fill it with the most relevant items first. Smaller context is usually cheaper, faster and more accurate.
• Filter by permissions before retrieval, so the model never receives data the user is not allowed to see.
• Split complex tasks into steps that each receive only the context for their part of the work.
• Test context changes against a fixed set of real examples, the same way code changes are tested.

Prompt engineering and context engineering

Prompt engineering remains part of the job. Context engineering adds the systems around the prompt that decide what reaches it. Anthropic's engineering team describes the shift in its guide to effective context engineering for AI agents, treating context as a limited resource with an attention budget.

The seven parts of the context

Each part competes for the same space. Tool definitions are a common hidden cost: an assistant connected to 40 tools sends 40 descriptions with every request, even when the task needs two.

Why a bigger context can make answers worse

Long context windows make it tempting to send everything. The evidence suggests otherwise. Chroma's Context Rot study, published in July 2025, tested 18 models and found that performance becomes less reliable as input length grows, even on simple retrieval tasks, and that models do not use their context evenly. Earlier research, known as Lost in the Middle, found that models use information at the beginning and end of a long input better than information in the middle.

For a business application this means three things. Irrelevant material is not neutral, since it competes with the facts that matter. The order of information matters, so the most important items belong near the start or end. And a long context costs more on every request, which adds up quickly at scale.

A context budget in practice

Consider an internal assistant that answers 20,000 questions a month about customer accounts.

The budgeted version costs about one twelfth as much, responds faster and usually answers more accurately, because the model is not searching a manual for the one paragraph that matters. The figures are illustrative, and the pattern holds across most business assistants.

A simple way to set the budget: decide the maximum tokens per request, reserve space for instructions and output, then fill the rest in order of relevance, starting with records and retrieved passages that directly concern the question.

Permission-aware context

In a business application, the context is also a data protection boundary. Anything placed in the context can appear in the answer, so the model must never receive information the user is not allowed to see.

• Filter before retrieval. Apply the user's access rights to the search query itself, so documents and records outside their permissions are never retrieved. Filtering after the model has seen them is too late.
• Pass identifiers, not datasets. Give the model the ID of the customer in question and a tool that fetches that customer's record with the user's permissions, rather than a list of customers.
• Mask what the task does not need. Payment details, health data and personal identifiers can often be removed or masked before they reach the model.
• Log what was sent. Record which documents and records each request included, so you can answer the question of what the model saw.

Our guide to an AI governance framework covers the rules on which data may reach which models.

Four context engineering examples from delivered projects

The cases below show what each step of the system receives. The prompt descriptions are simplified, but the context design is what the teams built.

Splitting one large prompt into five focused contexts

Before. A US personal injury law firm, Brandon J. Broderick, first tried a single ChatGPT prompt that received a whole case file of about 250 pages of medical records, police reports and notes, with a request to draft a legal opinion. It could not handle the volume or keep topics apart.

After. .wrk split the work across five specialized assistants. Each one receives only the questions for its topic, with its own examples, decision templates and instructions for edge cases, and returns structured output that follows the firm's template, with key conclusions in bold for attorney review. When an assistant got stuck on earlier turns, the session was reset through the API, which clears its context and starts it fresh. Model costs came to about $3–4 per case, and attorney throughput rose about 1.5 times. Read the BJB legal document processing case study.

Giving the model a list it must choose from

Before. An SEO assistant built into content management systems, made by a US marketing technology startup, asked a model to suggest internal links for an article. The model invented URLs that did not exist, and anchors did not match their destinations.

After. The context now includes an approved list of URLs for the model to pick, with rules for matching anchors to destinations and scoring relevance. Title suggestions received hard character limits and rules against keyword stuffing. The team built a list of failure types from production samples and ran regression checks on previously failing cases, so later model changes are tested against them. Hallucinated URLs stopped appearing in production. Read the SEO assistant case study.

Grounding generation in indexed content

Before. A content generation feature for a network of law firm websites risked producing articles that referred to sources nobody could verify.

After. .wrk built a Go service that searches the platform's own indexed content with a hybrid of keyword (BM25) and semantic (k-NN) search, plus a trend analyzer that scores topics. The generator receives the retrieved passages and trend data as its context, so its drafts are grounded in real, indexed sources. Read the Verba CMS case study.

Project instructions for a coding agent

Before. On a fast-moving threat intelligence platform for a US cybersecurity media company, the team had no shared practices for AI-assisted development, so coding assistants worked without knowledge of the project.

After. The team gave the assistant project-specific skills, meaning written instructions on the architecture, conventions and workflows it loads when relevant, ran it in a security sandbox and added automated code review in CI. The coding agent now starts each task with the project context it needs rather than guessing. Read the NEXUS case study.

Techniques for keeping context small and relevant

• Retrieval of passages. Split documents into sections, retrieve only the best matches and include their source, so answers can be checked. Our comparison of RAG and fine-tuning explains when retrieval is the right approach.
• Just-in-time loading. Give the model tools to fetch a record or document when it needs it, instead of loading everything in advance.
• Summarized history. Replace older conversation turns with a short summary and keep the last few messages in full.
• Structured state. For multi-step work, keep a short structured record of decisions and open items, and pass that instead of the full history.
• Task splitting. Break complex work into steps or sub-agents, each with its own focused context, and pass only the results forward.
• Trimmed tool results. Return only the fields the model needs from an API call, not the full response.
• Fewer tools per step. Show only the tools relevant to the current step. Our guide to MCP covers how tools are exposed to models.

How to test context changes

Context changes behave like code changes: they can fix one case and break another. Keep a fixed test set of 50–200 real requests with expected answers, including past failures. Run it whenever you change instructions, retrieval settings, tools or the model, and track accuracy, refusal rate, tokens per request and latency. Review a sample of production requests each week, including the context they received, to find new failure types.