Designing REST APIs LLMs Can Actually Use: A Field Guide for Tool-Using Agents

Every API team has a war story: they wired up their perfectly reasonable REST API to a tool-using agent, and the agent looped, hallucinated fields, called the wrong endpoint, or gave up. Then they blamed the model.

The model isn't the problem. Most APIs were designed for human developers who read docs, remember conventions, and have taste. Agents have none of those. They see a JSON schema and a description; they act on what those literally say.

Designing 'agent-friendly' APIs means removing every place where implicit knowledge is required. It also — happily — makes the API better for humans.

Short URLs are a legacy of typing on terminals. Agents don't type. Longer, explicit names reduce ambiguity dramatically.

Operation IDs in your OpenAPI spec become the tool names an agent sees. Make them read like verbs an intern would use: list_pending_invoices, not list_inv_p.

Every error should let the agent decide what to do next: retry, ask the user, or give up. That means three fields:

{
  "error": {
    "code": "customer_email_taken",
    "message": "A customer with this email already exists.",
    "next_action": "fetch_existing_customer",
    "next_action_params": { "email": "ada@example.com" },
    "retryable": false,
    "docs_url": "https://api.example.com/docs/errors/customer_email_taken"
  }
}

Agents retry. Networks fail. The user says 'try again'. Without an idempotency key, you get duplicate charges, duplicate emails, duplicate tickets.

Accept an Idempotency-Key header on every non-GET endpoint. Cache the response for 24h keyed by (endpoint, key, request-body-hash). Return the same response on retry.

Same pattern for webhooks going the other way — see webhooks that don't lie.

LLMs respect JSON Schema. Use it aggressively to prevent bad calls:

• Use enum for finite value sets — do not accept free-text where enums exist.
• Use format: 'email', format: 'date-time', pattern for regex constraints.
• Mark fields required when they truly are; agents skip optional fields aggressively.
• Add description on every field, in prose — this is what the model reads to know what the field means.
• Use oneOf for tagged unions with a discriminator; avoid anyOf which is ambiguous.

Agents lose track of page numbers across turns. Cursors are self-describing:

{
  "items": [...],
  "next_cursor": "eyJpZCI6MTIzNCwidHMiOiIyMDI2LTA3LTA4In0",
  "has_more": true
}

The agent just passes cursor back in the next call. No off-by-one, no state to remember, no double-fetching.

Polymorphic endpoints — POST /entities that accepts 12 different type values — confuse tool routers. Split into create_customer, create_invoice, create_ticket. Yes, it's more endpoints. Yes, it's better.

Your OpenAPI spec is the contract every tool-using framework reads. Treat descriptions as first-class UX:

paths:
  /customers:
    post:
      operationId: create_customer
      summary: Create a new customer record.
      description: |
        Creates a customer. Returns 409 if a customer with the same email
        already exists — in that case, call get_customer_by_email instead
        of retrying.
      requestBody:
        required: true
        content:
          application/json:
            schema: { $ref: "#/components/schemas/CustomerCreate" }

Tools like Chat GPT plugins, Anthropic tool use, and LangGraph ingest this spec verbatim. Bad descriptions == bad agent behavior.

Return 429 with headers the agent can act on:

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1720431600
Retry-After: 12

Agents will honor Retry-After automatically. Without it, they hammer you.

Before shipping API changes, run a smoke test with a real tool-using agent:

1. Feed the OpenAPI spec into your agent framework.
2. Give the agent a natural-language task ("create a customer named Ada with email x, then create an invoice for $100").
3. Log every call, every retry, every error.
4. Any confusion the agent shows is a doc bug or design bug — fix it, not the prompt.

Deeper eval methodology in evaluating LLMs.

APIs designed this way don't just work better for agents — they're easier for humans too. Better error messages, cleaner names, stricter schemas, and cursor pagination are best practices from long before LLMs existed. The pressure of tool-using agents just makes the tax on bad APIs finally visible.