OpenAI's new tools for building agents
This OpenAI article is useful as a baseline for Responses API, built-in tools, tracing, evaluations, and how agent platforms are getting structured.
Building an agent used to mean stitching together a model call, a vector store, a search API, a browser tool, and a custom orchestration layer. The Responses API tries to collapse all of that into one primitive — and the interesting part isn’t the API itself, but what it says about where agent platforms are converging.
Source
By: OpenAI
Published: Mar 11 2025
Type: Product announcement
This matters as a baseline for where agent platforms are heading. The Responses API, built-in tools, Agents SDK, and integrated tracing together define a pattern that other platforms are converging toward: model calls plus tools plus observability plus orchestration as one package.
How the Responses API structures the agent building blocks
The Responses API is OpenAI’s new primitive for building agents, replacing the Assistants API (deprecation targeted mid-2026) and extending the Chat Completions API. The core design: a single API call combines model inference with tool use, and as models get more capable, that single call can solve increasingly complex tasks using multiple tools across multiple model turns.
The key difference from Chat Completions is built-in tools. Three shipped at launch. Web search gives models fast, up-to-date answers with inline citations, powered by the same model behind ChatGPT search. On SimpleQA (factual accuracy benchmark), GPT-4o search scored 90%. File search retrieves from large document collections with query optimisation, metadata filtering, and custom reranking — a built-in RAG pipeline without external vector stores. Computer use lets the model generate mouse and keyboard actions, powered by the CUA model that also runs Operator. It scored 38.1% on OSWorld (full computer use), 58.1% on WebArena, and 87% on WebVoyager (web browsing).
The Responses API also introduces a unified item-based design with simpler streaming events, which makes consuming model output and tool-call results more predictable. The API stores data on OpenAI’s side (without training on business data by default), which enables the tracing and evaluation features.
The Agents SDK is the orchestration layer, open-source and built on top of Swarm. It provides configurable agents with instructions and tools, handoffs for transferring control between agents, guardrails for input/output validation, and tracing for visualising execution. A triage agent routes to specialised agents (shopping, support) based on the user’s intent — a pattern that works with both the Responses API and Chat Completions, and with models from other providers that support a Chat Completions-style endpoint.
Tracing and evaluations are the pieces that make this more than a calling convention. Without traces, a multi-step agent failure is a mystery. Without evaluations, changes to prompts and model choices are based on feeling rather than evidence. The article positions these as first-class, integrated features rather than afterthoughts — which matches what every production agent team eventually discovers they need.
The practical architecture that emerges: the Responses API handles model calls and built-in tools, the Agents SDK handles orchestration and handoffs, and tracing plus evaluations handle observability. Each layer is separable — you can use the API without the SDK, or the SDK with other models — but they’re designed to work best together.
The Computer Use tool is the most experimental piece. At 38.1% on OSWorld, OpenAI explicitly recommends human oversight for non-browser environments. The safety work includes prompt injection guards, confirmation prompts for sensitive tasks, environment isolation tools, and enhanced policy violation detection. This is a research preview, not a production-ready tool.
My take
The convergence is the story. Every agent platform is arriving at the same shape: model calls + built-in tools + tracing + evaluations + an orchestration SDK. What differs is whether these are integrated or assembled from separate vendors. OpenAI’s bet is that integration wins — one API, one data store, one tracing system. The trade-off is coupling: the closer you build to the Responses API’s assumptions, the harder it is to switch providers. The Agents SDK mitigating this by working with Chat Completions-compatible endpoints from other providers is a smart concession, but the built-in tools (web search, file search, computer use) lock you in by design. The right reading: use the Responses API as a reference architecture for what agent platforms need, but build your orchestration layer to be portable.
Where this is useful
- When starting a new agent project and choosing an API design — the Responses API is the clearest reference for how model calls, tool use, and streaming should fit together.
- When your agent fails on multi-step tasks and you can’t figure out where — integrated tracing is the feature that turns “it didn’t work” into “step 4 returned unexpected results from the file search tool.”
- When deciding whether to build evaluation into your agent workflow — the answer is always yes, and earlier is always better. Prompt changes without measurement are guesswork.
Apply this: If your agent doesn’t have tracing and evaluations yet, add them before your next model change. One measured improvement teaches you more about your system than ten prompt tweaks without data.
What I want to remember
Remember: An agent product should include evaluation and tracing early. Otherwise every improvement is a feeling instead of evidence. The platform converges on model + tools + traces + evals — build for that shape regardless of which provider you use.