Fix your AI agent from the dashboard, not the codebase.

Engineers add two lines and Agent Doctor captures every LLM call, vector lookup and tool call in the pipeline. Then anyone on the team can override a prompt and re-run it on your own servers, so the people who know the domain can fix the agent themselves.

Trace
support_agent9 steps1 failed3.21s
agenttasktoolretrieverllm
Auto-instruments the libraries you already import
Python SDK today, JavaScript/TypeScript next.

Override any prompt and re-run the whole pipeline on your servers.

A cardiologist with no coding background spots a wrong diagnosis, rewrites the faulty prompt from the dashboard, and re-runs the agent. The developer never gets paged.

MD
Cardiologist
Domain expert, no code access
Illustrative run
1Baseline Run (Auto-Captured)
agenttaskretrieverllm
Final Output (Incorrect)
"Patient likely has general anxiety. Recommend lifestyle changes and follow-up in 2 weeks."
Cardiologist: "This is wrong. These symptoms point to unstable angina, not anxiety. The prompt has no examples to guide it."
2Override from the dashboard
Selected: rank_diagnoses
System Prompt
"Analyze the symptoms and provide a diagnosis."
"Analyze the symptoms and provide a differential diagnosis. Use the examples below as guidance.

+ POSITIVE EXAMPLE:
Input: chest pressure, diaphoresis, jaw pain
Output: Unstable angina (ACS). Order troponin, 12-lead ECG.

- NEGATIVE EXAMPLE:
Input: chest pressure, diaphoresis, jaw pain
Output: General anxiety. Lifestyle changes. ← wrong"
Temperature
0.7  →  0.2
3Compare Results
Baseline run
DiagnosisGeneral anxiety
OverrideNone
Temperature0.7
Tokens286
Latency1.1s
Override run Override applied
DiagnosisUnstable angina (ACS)
Overridedifferential_dx
Temperature0.2
Tokens412+44%
Latency1.8s+0.7s
New Output (with few-shot examples)
"Presenting symptoms (chest pressure, diaphoresis, jaw pain radiating to left arm) are consistent with unstable angina. Recommend immediate troponin levels, 12-lead ECG, and cardiology consult."
Cardiologist: "Adding those two examples fixed it. The model needed to see what right and wrong look like."
Baseline captured

Developers build the pipeline and domain experts tune it.

An engineer can wire up the agent, but they can't judge a sound medical diagnosis or a legally airtight clause. Agent Doctor gives the people who can a dashboard to iterate on prompts and parameters, while the developer keeps ownership of the code.

Developer

Builds the pipeline

Writes the agent code, connects to databases, sets up tools, deploys to production.

def diagnose_patient(symptoms):
  history = ehr_db.lookup(patient_id)
  research = pubmed.search(symptoms)
  diagnosis = llm.generate(...)
  return treatment_plan(diagnosis)

Adds two lines & a webhook

Drops in the SDK. Done. Back to building features.

import agent_doctor
agent_doctor.init(api_key="sk-...")

Reviews & ships

When the domain expert finds a winning prompt, the dev reviews it and ships it with the next release.

←
traces
→
overrides
←
results
Domain expert

Sees every step visually

Opens the dashboard, sees the full pipeline trace. Every prompt, every output, every decision the agent made.

Tweaks prompts & parameters

Spots a bad prompt, rewrites it in plain English. Adjusts the temperature. Hits "Run". No code, no deploys.

- "Provide a general diagnosis"
+ "You are a board-certified cardiologist.
  Prioritize differential diagnosis.
  Cite relevant clinical guidelines."

Iterates until it's right

Compares runs side-by-side, uses Probe to debug, runs flakiness tests. All from the dashboard.

Example: Medical Triage Agent
Dev built
Doctor improved
DEV
Backend engineerBuilt the 6-node triage pipeline
Pipeline they builtintake → symptom_analysis → medical_db →
rank_diagnoses → treatment_plan → safety_review
Deployed & SDK initialized
MD
CardiologistIterates on prompts from the dashboard
What they changed (no code)Node: rank_diagnoses
Prompt: "Apply AHA guidelines for
 cardiac symptom evaluation..."

Temp: 0.7 → 0.2
Re-ran on the team's own servers, no deploy

Your existing code. Unchanged.

Agent Doctor hooks into LLM SDKs, vector databases, and tool calls at the library level. Your pipeline code stays exactly as-is. Every step is auto-captured.

support_agent.py
# Add these two lines, that's it
import agent_doctor
agent_doctor.init(api_key="sk-proj-...")

# Your existing agent code, zero changes needed
# LLM calls, vector DB queries and tool calls are auto-captured

def handle_ticket(query, customer_id):
    # LLM call → auto-captured
    intent = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": query}]
    )

    # Vector DB lookup → auto-captured
    docs = pinecone_index.query(vector=embed(query), top_k=5)

    # Tool call → auto-captured
    policy = check_refund_policy(customer_id)

    # LLM call → auto-captured
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "system", "content": f"Docs: {docs} Policy: {policy}"},
                  {"role": "user", "content": query}]
    )
    return response

# Your webhook handler. payload is the run Agent Doctor sends.
def on_run(payload):
    with agent_doctor.run_context(run_id=payload.run_id, overrides=payload.overrides):
        result = handle_ticket(payload.query, payload.customer_id)
        # Every step auto-captured. Overrides auto-injected. Done.

Two lines in, every step of your pipeline is in the dashboard.

Start free

Probe finds the broken node by asking you a few questions.

Tell it what the agent should have said and what it said instead. It walks the trace with you, asking three or four plain-English questions, and names the exact step that caused the failure. No logs, no code, no debugging skills needed.

  • A chat, so anyone on the team can run it
  • Checks the pipeline by halves, so it needs only a few questions
  • Highlights the broken node in the trace
  • Hands the fix straight to the prompt optimizer
Probe
Question 2
My agent should return a refund confirmation, but instead it's asking the customer to call support.
Got it. Let me trace through the pipeline. The router node received the customer's request and routed it to "refund_handler." Its output was: "Customer requests refund for order #4821." Does this routing look correct?
Yes, that's right.
Good. The refund_handler then looked up order #4821 and produced: "Order found. Status: delivered. Refund policy: eligible within 30 days." Does this look correct?
Yes, that's correct too.
Root cause found: response_composer

All upstream nodes are healthy. The response_composer node received correct refund eligibility data but generated a "call support" response instead of processing the refund.

Suggestion: The prompt lacks explicit instruction to confirm eligible refunds directly.

Open in Prompt Optimizer →

Flakiness detection finds the step that answers differently every time.

Run your pipeline ten times with identical input. Per-node consistency scores show exactly which step is non-deterministic, and an AI report tells you why.

Stability Analysis
10 / 10 runs
user_query0.99
→
router0.54
→
rag0.78
↓
synthesize0.72
→
validate0.98
→
response0.95
FLAKY router | picked a different route in 3 of 10 replays
  7/10 → Knowledge Base  |  3/10 → Account Tools
MODERATE synthesize | Consistency: 0.72
STABLE validate | Consistency: 0.98

Root cause: Router prompt lacks explicit routing rules for password queries.
  • Per-node consistency scores and decision divergence
  • AI root cause report with a suggested fix
  • Apply the fix, re-test, and verify from the same screen

The optimizer writes, tests and ranks better prompts for you.

Pick a node and define what "good" looks like. The optimizer generates prompt variants, runs them against your examples, and ranks the results. It starts with GEPA, a genetic strategy that evolves prompts across generations, with more strategies on the way.

  • Evolves prompts across generations and keeps the fittest
  • Rephrases, expands, compresses and recombines winning traits
  • Ranks every variant by its score on your examples, with token cost weighed in
  • Shows the winning prompt side by side with the baseline
Prompt Optimizer
GEPA, generation 7 of 7
GEPA
Illustrative run
1.0
0.65
0.3
#VariantFitnessHoldoutTok.
1C140.940.91134
2C110.920.89151
3C90.890.87168
BBaseline0.710.70185
Role: Elite
Task: Rephrased
Format: Crossover
Constraints: Negated
Examples: Elite

Why not just use a regular observability tool?

Most LLM tracing platforms run experiments on their own cloud, so your agent can't reach your private databases, RAG, or internal tools during a test. They also keep prompt iteration locked behind code, leaving your domain experts out.

Capability
Hosted tracing tools
Agent Doctor
Re-runs reach your private DBs, RAG and tools
Experiments run in their cloud, away from private infra
Re-runs execute on your servers
Domain experts tune prompts without code
Usually a developer scripts the runs
Visual override editor
Root-cause the exact node behind a bad output
Mostly read the trace yourself
Probe
Evolve better prompts automatically
Varies by tool
Prompt Optimizer (GEPA)
Setup effort
SDK wrappers & decorators
Two lines, no wrappers

Common questions

The trace, and only the trace. Re-runs use remote-triggered execution: your webhook runs the agent inside your own infrastructure, where it already reaches your private databases, vector stores and internal tools, and Agent Doctor never connects to any of them. What it receives is the trace your SDK sends: each step's inputs, outputs and timings, which is what the dashboard shows you.

Two lines: import the SDK and call init(). It auto-instruments your LLM calls, vector DB lookups, tool executions, and API requests at the library level, with no decorators, wrappers, or refactors required.

A Python SDK today, with JavaScript/TypeScript next. It auto-instruments the libraries listed at the top of this page, from OpenAI and Anthropic to LangChain, LlamaIndex and the major vector databases.

Yes. Domain experts get a visual dashboard to inspect every step, rewrite prompts in plain English, adjust the temperature, and re-run, all without opening the codebase. Developers stay focused on building.

The free plan covers one project, 5 runs, 10 playground calls and 3 Probe sessions a month, with no card required. Flakiness detection and prompt optimization start on Starter. Paid plans raise every limit, and each triggered run counts against your plan, so you pay for what you actually test. Reach out for team and enterprise plans.

Ready to diagnose and fix?

Install the SDK, open the dashboard, and let the whole team work on the agent.

No card required 5 runs a month free Two lines of SDK code