Prompt Engineering for Agents: A Structured, Testable, Version-Controlled Approach

In 2023 prompt engineering was a party trick. In 2026 it's a shipping discipline: versioned in Git, validated with schemas, gated by evals, rolled out gradually, and rolled back in one click. The teams that treat prompts like code ship 3× more experiments per quarter and see 70% fewer prompt-related incidents. This article is the full workflow.

Every production prompt I ship has the same five blocks, in this order. The order matters — models pay more attention to the top and bottom of the context window, so instructions live at the top and constraints repeat at the bottom.

1. Role — who the model is, whose interests it serves, what it optimizes for.
2. Context — retrieved data, user profile, tool schemas, conversation history.
3. Task — the concrete request, one imperative sentence.
4. Constraints — output schema, forbidden actions, tone, refusal policy.
5. Examples — 2–4 few-shots covering easy, hard, and refusal cases.

<role>
You are a support triage assistant for Acme SaaS. Optimize for correctness
and safe escalation over speed. You never invent policy.
</role>

<context>
{{retrieved_policy_chunks}}
Customer: {{customer.name}} · Plan: {{customer.plan}} · Region: {{customer.region}}
</context>

<task>
Classify the ticket and draft a one-paragraph reply.
</task>

<constraints>
- Output ONLY the JSON matching the tool schema. No prose outside it.
- If unsure, set escalate=true and copy the exact policy passage into reason.
- Never mention prices or refunds not present in <context>.
</constraints>

<examples>
{{fewshots}}
</examples>

Blocks are separated by clear markers. This makes diffs readable and lets you swap one block without rewriting the rest — critical for A/B testing constraint tweaks or few-shot changes independently.

Never parse free text if you can help it. Use JSON mode, function calling, or grammar-constrained decoding with a schema. The schema is not just runtime validation — it's the API contract, the eval spec, and the documentation, all in one file.

{
  "name": "classify_ticket",
  "parameters": {
    "type": "object",
    "required": ["intent", "urgency", "escalate"],
    "properties": {
      "intent": { "enum": ["refund", "bug", "how_to", "sales", "unknown"] },
      "urgency": { "enum": ["low", "med", "high"] },
      "escalate": { "type": "boolean" },
      "reason":   { "type": "string", "maxLength": 200 },
      "citations": {
        "type": "array",
        "items": { "type": "string" },
        "description": "IDs of chunks from <context> that support the classification."
      }
    }
  }
}

Validate every response against the schema before it leaves the agent. On validation failure, do one automatic repair pass (re-ask the model with the schema errors) and DLQ anything that still fails. This single pattern eliminates ~90% of downstream JSON parsing bugs.

The average team throws in 8 random examples and hopes. The good ones pick 3–4 examples that maximize diversity of decision boundary: one obviously easy, one hard-but-decidable, and one that must be refused. If two examples teach the same lesson, drop one.

• Cover every enum value in the output schema at least once across your few-shots.
• Include one near-miss where the naïve model would pick the wrong enum — this is where few-shots earn their keep.
• Always include one abstention example (unknown intent, missing data) so the model knows refusal is allowed.
• Rotate few-shots based on the retrieved context if you can — dynamic few-shots outperform static ones on domain-specific tasks.

A prompt registry is a tiny Postgres table plus a 50-line rendering helper. It replaces the entire class of 'we redeployed the app to change one word' incidents.

create table prompts (
  id uuid primary key default gen_random_uuid(),
  name text not null,
  version int not null,
  template text not null,
  schema jsonb not null,
  model text not null,
  temperature real not null default 0,
  is_active boolean not null default false,
  created_at timestamptz not null default now(),
  unique (name, version)
);
create unique index one_active_per_name
  on prompts (name) where is_active;

The one_active_per_name partial unique index enforces the golden rule: exactly one active version per prompt name. Rollback is a single SQL statement: flip is_active from the bad version to the previous good one. No redeploy, no rebuild.

The runtime helper fetches the active prompt by name, caches it for 60 seconds, renders the template with the current context, and logs the version used with every request. Reviewing an incident becomes trivial: filter logs by prompt_name = X and version = Y.

• Store the template as a Git-committed file — the registry mirrors from Git via CI, not the other way around. Git is the source of truth; the DB is the runtime.
• PRs must include the diff plus the eval delta. If unit or regression evals drop, the PR is blocked, not debated.
• Tag releases (v1.4.0) and record which release served each production request in your logs.
• Ship behind a feature flag: 5% traffic → 25% → 100% over 24–48h. Monitor grounding, refusal rate, tool-call error rate, and cost per request during each stage.

The reason most prompt changes ship late and break is that nobody has an eval. Build three suites — small, cheap, and progressively slower — and gate merges on them.

Unit tests are hand-written assertions on structured output (intent = refund, urgency = high). Regression is a sample of real requests replayed with the new prompt, scored by an LLM judge. Adversarial is a growing library — every incident postmortem adds a case.

A judge prompt is itself a prompt: it deserves the same five-block skeleton, structured output, and evals against human labels. Rules that work in practice:

• Score one dimension at a time (grounding, tone, refusal-correctness). Multi-dimensional single-shot scores are noisy.
• Use a small, fast model (Haiku, gpt-4o-mini) for scale, and reserve the strongest model for spot audits.
• Compare against a human-labeled gold set quarterly; if agreement drops below 85%, retire the judge and retrain.
• Always store the judge's raw output alongside the score — future you will need it to debug drift.

<role>
You are a strict evaluator scoring answer GROUNDING only. You do not care
about tone, style, or completeness. You care about one thing: does every
factual claim in the answer appear in the provided context?
</role>

<constraints>
Output JSON: { "grounded": bool, "unsupported_claims": string[] }.
</constraints>

Prompt cost is a design output. Bake three levers into every prompt so you can tune cost without rewriting logic:

1. Model tier in metadata — swap Haiku ↔ Sonnet ↔ Opus without touching the template.
2. Context budget — a hard token cap on {{retrieved_chunks}}. Reranker + cap beats bigger context windows for both cost and grounding.
3. Cache key — the deterministic part of the prompt (role, task, constraints, few-shots) hashed as a prefix. Providers like Anthropic and OpenAI charge less for cache hits; use them.

The result: a 10× cost swing between the cheap and premium path for the same prompt, controlled by config not code.

1. Day 1–2: pick one prompt, rewrite in the five-block skeleton, add a JSON schema.
2. Day 3–4: hand-write a 30-example unit eval set. Get it to 100% on the new prompt.
3. Day 5: stand up the prompts registry table + a 50-line renderer.
4. Day 6–7: move the prompt into the registry, ship behind a 5% flag, monitor 48h.
5. Week 2 · Day 1–2: sample 300 production traces into a regression set, score with an LLM judge you validated against 50 human labels.
6. Week 2 · Day 3–4: build the adversarial set from your last 20 incident postmortems.
7. Week 2 · Day 5: wire evals into CI — unit blocks merge, regression comments on PR, adversarial runs weekly.
8. By day 10: every new prompt change ships through the same pipeline; rollback is one SQL statement.