Prompting, tool use and agents

Overview

A language model on its own produces text. A useful system wraps it in a prompt that says what is wanted, a parser that checks what came back, tools the model can ask to have run, and a loop that keeps going until the task is done or a budget is spent. This hub is about that wrapping, from a single well-specified prompt to a multi-step agent that reads files, calls APIs and edits code. It is for engineers building on hosted models from OpenAI, Anthropic and Google or on open models served locally, since the patterns are the same.

It matters because the model is the smallest part of the engineering. The difference between a demo and a product is in the prompt’s precision, the validation of outputs, the limits placed on the loop, and the evaluation that catches regressions when the model or the prompt changes.

Key ideas

A prompt is a specification

Treat a prompt as you would a function signature and its docstring. State the role, the task, the constraints, the output format and what to do when the input is unusable, in that order. Give two or three worked examples when the format is subtle, since examples are read more reliably than rules. Put stable instructions in the system prompt and the per-request material in the user turn, which also lets providers cache the stable prefix and cut cost. Reasoning models, which generate a hidden chain of thought before answering, want the goal and the constraints and little else, while older models benefit from being asked to work step by step. Version prompts in git with the model name and date, because a prompt tuned for one model release will behave differently on the next.

Structured output and tool calling

When the output feeds a program, ask for JSON against a schema and validate it. Providers now offer constrained decoding, which guarantees the output parses, and libraries such as Pydantic and Instructor validate the content and retry with the error message when a field is wrong. Tool calling (function calling) is the same idea in the other direction: you describe functions with a name, a purpose and a JSON schema for the arguments, and the model answers with a call rather than prose. Your code runs the function and sends the result back as a new message. The model never executes anything itself, which is both the safety property and the reason the loop below exists.

The agent loop

An agent is a model, a set of tools and a loop. Each turn the model sees the conversation so far and either answers or asks for a tool. The loop runs the tool, appends the observation and calls the model again. Everything that makes agents hard follows from this shape: the context grows every turn, a wrong observation misleads every later step, and without a stop condition the loop runs until the budget is gone.

The Model Context Protocol (MCP), published by Anthropic in November 2024 and since adopted by OpenAI, Google and most agent frameworks, standardises the tool side. A server exposes tools, resources and prompt templates, a client discovers them, and the model sees them as ordinary function schemas. The practical benefit is that a file system, a database or a ticketing system is wrapped once and used from any agent.

Security belongs in the loop design, not in the prompt. Anything a tool returns, a web page, an email, a file, can contain text that reads as an instruction, and the model cannot reliably tell it from yours. Give each tool the least access it needs, require confirmation for destructive actions, keep secrets out of the context, and log every call. The safety hub covers prompt injection in depth.

Evaluating agents

Reading three transcripts is not an evaluation. Build a set of tasks with a checkable end state, run the agent on each several times, and report the success rate with cost and latency per task. Public benchmarks such as SWE-bench (resolving real GitHub issues) and tau-bench (customer-service tasks with a simulated user) show the shape: the score is whether the final state is right, not whether the transcript looks sensible. Keep traces for every run so that a regression can be traced to a prompt change, a model update or a tool failure. Where the output is free text, an LLM judge with a rubric is acceptable if you calibrate it against human labels on a sample first. See the evaluation hub.

Where to start

Begin with Using an LLM well, which covers prompts, structure and limits, then From notebook to service for the wrapper around the model. Embeddings and similarity search prepares you for the retrieval tool most agents need. The prompt patterns and structured output reference is the cheat sheet, and the RAG hub covers the most common tool.

Tools and platforms

Common mistakes

• Offering forty tools with vague descriptions. Tool choice degrades with count, so group tools by task and describe each in one precise sentence.
• Running a loop with no turn or token budget. The failure mode is a 400 USD bill from a model that could not find a file.
• Treating tool output as trusted. A fetched page that says “ignore previous instructions” is an attack, and the loop must assume it.
• Parsing JSON without validating its content. A field can be syntactically valid and semantically wrong.
• Shipping a prompt change with no regression suite, then discovering the format broke on 5 percent of inputs.
• Testing at temperature 0 once and calling it deterministic. Run each evaluation task several times and report the spread.

Going deeper

• Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, 2022.
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, 2022.
• Greshake et al., Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection, 2023.
• Jimenez et al., SWE-bench, 2023, and Yao et al., tau-bench, 2024.
• The Model Context Protocol specification.
• The Anthropic tool use documentation and the OpenAI function calling guide.