Know how your agent behaves before it ships.
CompFly simulates adversarial prompts, unsafe tool use, data leakage, model drift, delegation failures, and workflow edge cases before agents reach production. Every finding becomes evidence your teams can review, act on, and retest after each change.
What ships when nobody attacks first.
The failures that end up in an incident report are the ones no one probed for. Simulation probes for them on purpose, before and after go-live.
- Blind deployment, teams put agents into production without knowing how they fail under attack.
- Upgrade risk, an agent or model upgrade quietly reintroduces weaknesses that were already fixed.
- Post-launch regressions, risk does not stop at launch; changes after go-live slip in untested.
- No sign-off gate, there is no security gate an agent has to pass before it ships.
- Hidden over-refusal, an agent that refuses everything passes no real test, and the metrics never show it.
Two simulations for two kinds of failure.
Some failures need an attacker. Others happen with ordinary users on an ordinary day. Agent Simulation tests both.
Security simulation
Assumes an active attacker. Works the same surfaces a real one would, then reports what got through and what held.
Findings map to OWASP Agentic AI Top 10 · MITRE ATLAS.
Safety simulation
No attacker in the loop. Measures how the agent behaves with ordinary users: what it makes up, who it treats differently, and what it wrongly refuses.
Maps to recognized AI risk frameworks.
Security and safety scores per agent, tracked run over run, regressions show up as a dip, not a surprise.
Three depths, from smoke test to threat simulation.
Need domain-specific coverage? Bring your own scenarios and datasets and run them alongside the generated ones.
Quick
Standard
Threat Simulation
Find the exact point where defenses break.
Attacks escalate through sophistication levels, L1 to L4. When the agent blocks a technique at a low level, the run does not move on, it retries harder until it finds the defense ceiling: the level where that defense gives way. Many agents hold the line on the first message but erode over several turns, so multi-turn escalation is tested too.
A real Threat Hunt report: technique-by-sophistication coverage against MITRE ATLAS, down to the cell.
Tests the agent you actually run.
Not a generic benchmark. Every run starts from what this agent can really do.
Per-run profile
Discovered, scored, compared
Each run discovers the agent’s actual tools, retrieval, memory, and model configuration. Every tool gets a risk level; attack surfaces are counted. Profiles are compared across runs, a new tool, or a tool whose risk level changed, is flagged as drift before it is found in an incident.
Instrumented runs
Payloads at every layer
With runtime instrumentation in place, adversarial payloads are injected at the model, tool, retrieval, memory, and error-handling layers during live execution; the report shows where defenses held, layer by layer. Without instrumentation, the run falls back to text-only evaluation.
Findings that become controls.
Every finding carries a prioritized recommendation, from “the agent leaked data” down to the exact change that stops it. Findings do not stop at a report, they feed Runtime Governance, so a weakness found here becomes a control enforced in production. The loop closes without a handoff document.
Defense rate
How much of the escalating attack set the agent defended, weighted by how hard each technique was pushed.
Trust score
How far the agent can be trusted on hallucination, bias, refusal, and boundary adherence.
Simulation runs continuously, not once.
A test that ran last quarter says nothing about the agent running today.
Scheduled & event-driven
In CI/CD
Around the deploy
See your agent’s defense ceiling.
We will run Agent Simulation against one of your agents and show where it holds, where it erodes, and the exact changes that fix it.