Stoa · case study

Nine agents. Three criticals. One scan.

We built a realistic multi-agent customer-support pipeline — a LangGraph supervisor routing to five specialist agents across four frameworks, plus a TypeScript chat widget — planted the kinds of risks that appear in real codebases, and pointed the published stoa package at it. This is what came back, finding by finding.

The system under test

A support desk the way teams actually build them: one orchestrator, specialist agents with real powers, shared database tooling, and a web-tier front door.

examples/support-desk · 14 source files
                    ┌──────────────────┐
customer chat ────► │  orchestrator.py │  LangGraph supervisor
                    └────────┬─────────┘
      ┌──────────────┬───────┴───────┬────────────────┐
      ▼              ▼               ▼                ▼
triage_agent   billing_agent   account_agent   escalation_agent
PydanticAI     LangChain       CrewAI          raw Anthropic
intent router  Stripe+Postgres Postgres+SendGrid Zendesk+Slackresearch_agent  OpenAI Agents SDK + Tavily

shared: tools/database.py (Postgres+Redis) · tools/diagnostics.py (subprocess)
        web/support_widget.ts (TypeScript front door)

What Stoa reported

From stoa scan examples/support-desk — the full report is live here, byte-for-byte what the tool produced.

14
files scanned
9
agent candidates
5
high confidence
3
critical findings
27
review prompts
1
suppressed, with reason
SeverityRuleLocationWhat it means in this pipeline
critical SEC001 billing_agent.py:16 A hardcoded “temporary” API key on the agent that moves money. High confidence — this is the one finding that would fail a PR gate.
critical SEC002 account_agent.py:15 A hardcoded production DB password on an agent that can write to customers and send email — change the address, send the reset link: an account-takeover toolkit.
high SEC003 billing_agent.py:38 Customer ID interpolated into SQL. With an LLM choosing tool inputs, a prompt-injected “my id is 1 OR 1=1” becomes an exfiltration attempt.
medium REL001 escalation_agent.py:44 The on-call Slack ping swallows every exception — escalations can vanish silently.
medium NET001 research_agent.py:31 Plain-HTTP intranet knowledge base, on the agent that ingests web content — the pipeline’s prompt-injection ingress.
medium NET002 billing_agent.py:43 Ledger sync with no timeout: one hung internal service stalls the refund path.
info ×27 CTRL001–003 one per agent Auth, input validation, and rate limiting were not observed in these files — review prompts, never claimed as vulnerabilities, never gating.

The headline risk isn’t a line of code

It’s a pairing

Stoa’s exposure ranking put billing_agent first not because of any single finding, but because one candidate combines payment_access + tool_calling + database_read: an agent that both decides and moves money. The account agent ranks second for the same reason — database_write + email_send is an identity-takeover surface.

That’s the point of an agent inventory: individual findings get fixed, but capability pairings are architectural — and invisible unless something maps them.

Just as important: what didn’t fire

A scanner you trust is defined by its silences.

FileOutcomeWhy it matters
embeddings.pynot flaggedOpenAI import + API call, but embeddings-only — weighted evidence says “not an agent.”
utils/user_agent.pynot flaggedUserAgentParser — generic *Agent names are weak evidence.
tests/test_billing.py:3low confidenceA placeholder key in a test — reported honestly, but can never fail a build.
tools/database.py:40suppressedA reviewed, enum-driven query — suppressed with a reason, still counted and listed.
parameterized queriesno noise%s placeholders throughout produce zero SEC003 findings.

The scan that fixed the scanner

Found: one false positive. Shipped: 0.1.1, same day.

The stress test flagged the escalation agent’s Zendesk call as missing a timeout — but the timeout was there, on the fifth line of the call, just past Stoa’s 4-line look-ahead. The window is now 8 lines, a regression test pins it, and v0.1.1 shipped the same day.

That’s what this case study is for: the demo stress-tests the scanner as much as the scanner audits the demo.

Run it on your own pipeline

Two commands
$ pipx install stoa-agent-risk
$ stoa scan . && open stoa-report.html

Your report will look exactly like the live scan report — but for your agents.