How evaluations work
Overview
An evaluation measures how an agent behaves when someone tries to misuse it. Relyance maps the agent, generates tests for each possible attack, runs the tests many times in a sandbox, and records what the agent did in each run.
Attack surface
An evaluation starts with the agent's attack surface. Relyance identifies:
- the tools that the agent can call
- the data stores that the agent reads, and the sensitive data in them
- the untrusted inputs, which are places where the agent reads text that an outside party wrote
Relyance then finds the attack paths. An attack path is a route from an untrusted input to a sensitive action or to sensitive data. For example: the agent reads a note that a customer wrote, and the agent has a tool that sends email.
You confirm the facts that Relyance cannot read from the agent's configuration. See Onboard an agent.
Test cases
Relyance generates test cases for each attack path. Each test case has two parts:
- An attack. An attempt to make the agent follow that attack path. Some attacks use one message. Others use a conversation with several turns.
- A normal task. A legitimate request that a user could make in the same situation.
The two parts run together. This shows whether the agent resists the attack and whether it still completes the user's task.
Attacks belong to four categories:
| Category | Goal of the attack |
|---|---|
| Prompt injection | Put instructions in content that the agent reads, so that the agent follows them |
| Data exfiltration | Move sensitive data out through a reply or a tool |
| Abuse | Use the agent for a purpose that it is not intended for |
| Privilege escalation | Make the agent take an action that the requester is not allowed to take |
Datasets
Test cases are grouped into datasets:
| Dataset | Source |
|---|---|
| Relyance Security Index | Generated from the agent's attack surface. Always included. It gives a common measure across your agents and models. |
| From logs | Sessions that you promote from production traces. Normal sessions become tasks. Risky behavior becomes attack test cases. |
| Custom evals | Rules that you describe in plain language. Relyance generates the test cases and the scoring for each rule. |
The sandbox
Each run takes place in an isolated sandbox:
- The agent uses its real instructions and tool definitions.
- Tools are simulated. When the agent sends an email or queries a table, no real system is called.
- Data is synthetic. It has the same structure as your data, but it is not a copy of your data.
- Each run uses a new environment, so one test cannot affect another.
An agent can respond differently to the same input, so each test case runs several times. Each test case also runs in two configurations: without Guardrail, which shows the agent's behavior with no protection, and with Guardrail, which shows the effect of protection.
Scores
Relyance reports three scores:
| Score | What it measures | Pass condition |
|---|---|---|
| Successful attacks | How often an attack achieved its goal | With Guardrail, one tenth or less of the count without Guardrail |
| Tasks completed | How often the agent completed the user's task | 99% or more of normal tasks, with Guardrail |
| Privilege escalations | How often the agent took an action beyond the requester's permissions | None without approval |
Each score has a Pass, Warn, or Fail result. Relyance shows each score with and without Guardrail.
Scores are based on the agent's actions: the tools it called, the arguments it sent, and the data in its replies. A run does not pass because the agent's reply says that it refused.
Experiments
An experiment runs the full dataset one time, with and without Guardrail. Relyance compares each experiment with a baseline. If a score moves out of its pass range, Relyance marks the experiment as a regression.
Use experiments to:
- Track changes. Run an experiment for each commit or model and compare the scores over time.
- Compare models. Run the same dataset on other models and compare security and cost.
- Set merge requirements. Require the scores to pass before a pull request can merge.
- Inspect a run. Open a result to see the trace of that run: each model call, each tool call, and the step where the attack succeeded or was stopped.
If you decide to accept the risk that a test case shows, set its status to Risk accepted. The test case then has a neutral score, and Relyance records who accepted the risk.
Benchmarks
The Overview tab shows two kinds of results. The Relyance Security Index evaluates your agent in its sandbox. Public benchmarks evaluate the model in their own environments, and Relyance shows them for reference with a confidence interval. The first measures the agent. The second measures only the model.
Reports
| Report | Content |
|---|---|
| Executive summary | Security Index, release gate result, and top findings |
| Technical report | Each finding with evidence and mitigation |
| OWASP LLM Top 10 | Test coverage and findings for each risk |
| Compliance evidence | How the evaluation supports adversarial testing requirements in the EU AI Act, ISO/IEC 42001, and the NIST AI RMF |
| Runs (CSV) | Each test on each configuration |