# How evaluations work

## Overview

An evaluation measures how an agent behaves when someone tries to misuse it. Relyance maps the agent, generates tests for each possible attack, runs the tests many times in a sandbox, and records what the agent did in each run.

<figure style="margin:1.5rem 0">
<svg viewBox="0 0 880 400" width="100%" role="img" aria-labelledby="aig-eval-title aig-eval-desc" style="max-width:880px;height:auto;font-family:var(--body)">
<title id="aig-eval-title">Anatomy of an evaluation</title>
<desc id="aig-eval-desc">Each test case pairs an attack with an everyday task. Both enter an isolated sandbox that contains the agent, simulated tools and synthetic data. The test runs several times without Guardrail and several times with Guardrail. The runs produce three scores: successful attacks, tasks completed and privilege escalations.</desc>
<marker id="aig-eval-arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0 0L10 5L0 10z" style="fill:var(--accent)"/></marker>
<text x="20" y="28" style="fill:var(--ink);font-size:15px;font-weight:700">Test case</text>
<rect x="20" y="42" width="200" height="70" rx="10" style="fill:var(--danger-tint);stroke:var(--danger-line)"/>
<text x="120" y="72" text-anchor="middle" style="fill:var(--ink);font-size:15px;font-weight:700">Attack</text>
<text x="120" y="93" text-anchor="middle" style="fill:var(--muted);font-size:12px">Aimed at one attack path</text>
<rect x="20" y="126" width="200" height="70" rx="10" style="fill:var(--ok-tint);stroke:var(--ok-line)"/>
<text x="120" y="156" text-anchor="middle" style="fill:var(--ink);font-size:15px;font-weight:700">Everyday task</text>
<text x="120" y="177" text-anchor="middle" style="fill:var(--muted);font-size:12px">The work the user wanted</text>
<rect x="290" y="20" width="330" height="220" rx="12" style="fill:var(--accent-tint);stroke:var(--accent);stroke-width:1.5"/>
<text x="310" y="46" style="fill:var(--ink);font-size:15px;font-weight:700">Isolated sandbox</text>
<rect x="310" y="60" width="290" height="50" rx="8" style="fill:var(--panel);stroke:var(--line)"/>
<text x="455" y="90" text-anchor="middle" style="fill:var(--ink);font-size:14px">Your agent: model, instructions, tools</text>
<rect x="310" y="122" width="138" height="50" rx="8" style="fill:var(--panel);stroke:var(--line)"/>
<text x="379" y="152" text-anchor="middle" style="fill:var(--ink);font-size:14px">Simulated tools</text>
<rect x="462" y="122" width="138" height="50" rx="8" style="fill:var(--panel);stroke:var(--line)"/>
<text x="531" y="152" text-anchor="middle" style="fill:var(--ink);font-size:14px">Synthetic data</text>
<text x="455" y="200" text-anchor="middle" style="fill:var(--muted);font-size:12px">Repeated runs, without Guardrail and with Guardrail</text>
<text x="455" y="220" text-anchor="middle" style="fill:var(--muted);font-size:12px">No production data. No credentials. No side effects.</text>
<path d="M220 77H288" style="fill:none;stroke:var(--accent);stroke-width:1.5" marker-end="url(#aig-eval-arrow)"/>
<path d="M220 161H288" style="fill:none;stroke:var(--accent);stroke-width:1.5" marker-end="url(#aig-eval-arrow)"/>
<text x="680" y="28" style="fill:var(--ink);font-size:15px;font-weight:700">Every run is recorded</text>
<rect x="680" y="42" width="180" height="154" rx="10" style="fill:var(--panel);stroke:var(--line)"/>
<text x="770" y="76" text-anchor="middle" style="fill:var(--ink);font-size:14px">Each model call</text>
<text x="770" y="104" text-anchor="middle" style="fill:var(--ink);font-size:14px">Each tool call</text>
<text x="770" y="132" text-anchor="middle" style="fill:var(--ink);font-size:14px">Each reply</text>
<text x="770" y="168" text-anchor="middle" style="fill:var(--muted);font-size:12px">Open any run as a trace</text>
<path d="M620 120H678" style="fill:none;stroke:var(--accent);stroke-width:1.5" marker-end="url(#aig-eval-arrow)"/>
<path d="M455 240V282" style="fill:none;stroke:var(--accent);stroke-width:1.5" marker-end="url(#aig-eval-arrow)"/>
<rect x="110" y="286" width="210" height="90" rx="10" style="fill:var(--panel);stroke:var(--line)"/>
<text x="215" y="320" text-anchor="middle" style="fill:var(--ink);font-size:15px;font-weight:700">Successful attacks</text>
<text x="215" y="344" text-anchor="middle" style="fill:var(--muted);font-size:12px">Did the attack get through?</text>
<rect x="350" y="286" width="210" height="90" rx="10" style="fill:var(--panel);stroke:var(--line)"/>
<text x="455" y="320" text-anchor="middle" style="fill:var(--ink);font-size:15px;font-weight:700">Tasks completed</text>
<text x="455" y="344" text-anchor="middle" style="fill:var(--muted);font-size:12px">Did real work still get done?</text>
<rect x="590" y="286" width="210" height="90" rx="10" style="fill:var(--panel);stroke:var(--line)"/>
<text x="695" y="320" text-anchor="middle" style="fill:var(--ink);font-size:15px;font-weight:700">Privilege escalations</text>
<text x="695" y="344" text-anchor="middle" style="fill:var(--muted);font-size:12px">Did it exceed its authority?</text>
</svg>
</figure>

## Attack surface

An evaluation starts with the agent's attack surface. Relyance identifies:

- the tools that the agent can call
- the data stores that the agent reads, and the sensitive data in them
- the untrusted inputs, which are places where the agent reads text that an outside party wrote

Relyance then finds the **attack paths**. An attack path is a route from an untrusted input to a sensitive action or to sensitive data. For example: the agent reads a note that a customer wrote, and the agent has a tool that sends email.

You confirm the facts that Relyance cannot read from the agent's configuration. See [Onboard an agent](/docs/evaluations-and-guardrails/onboard-an-agent/).

## Test cases

Relyance generates test cases for each attack path. Each test case has two parts:

- **An attack.** An attempt to make the agent follow that attack path. Some attacks use one message. Others use a conversation with several turns.
- **A normal task.** A legitimate request that a user could make in the same situation.

The two parts run together. This shows whether the agent resists the attack and whether it still completes the user's task.

Attacks belong to four categories:

| Category | Goal of the attack |
|---|---|
| **Prompt injection** | Put instructions in content that the agent reads, so that the agent follows them |
| **Data exfiltration** | Move sensitive data out through a reply or a tool |
| **Abuse** | Use the agent for a purpose that it is not intended for |
| **Privilege escalation** | Make the agent take an action that the requester is not allowed to take |

## Datasets

Test cases are grouped into datasets:

| Dataset | Source |
|---|---|
| **Relyance Security Index** | Generated from the agent's attack surface. Always included. It gives a common measure across your agents and models. |
| **From logs** | Sessions that you promote from production traces. Normal sessions become tasks. Risky behavior becomes attack test cases. |
| **Custom evals** | Rules that you describe in plain language. Relyance generates the test cases and the scoring for each rule. |

## The sandbox

Each run takes place in an isolated sandbox:

- The agent uses its real instructions and tool definitions.
- Tools are **simulated**. When the agent sends an email or queries a table, no real system is called.
- Data is **synthetic**. It has the same structure as your data, but it is not a copy of your data.
- Each run uses a new environment, so one test cannot affect another.

An agent can respond differently to the same input, so each test case runs several times. Each test case also runs in two configurations: **without Guardrail**, which shows the agent's behavior with no protection, and **with Guardrail**, which shows the effect of protection.

## Scores

Relyance reports three scores:

| Score | What it measures | Pass condition |
|---|---|---|
| **Successful attacks** | How often an attack achieved its goal | With Guardrail, one tenth or less of the count without Guardrail |
| **Tasks completed** | How often the agent completed the user's task | 99% or more of normal tasks, with Guardrail |
| **Privilege escalations** | How often the agent took an action beyond the requester's permissions | None without approval |

Each score has a **Pass**, **Warn**, or **Fail** result. Relyance shows each score with and without Guardrail.

Scores are based on the agent's actions: the tools it called, the arguments it sent, and the data in its replies. A run does not pass because the agent's reply says that it refused.

## Experiments

An **experiment** runs the full dataset one time, with and without Guardrail. Relyance compares each experiment with a baseline. If a score moves out of its pass range, Relyance marks the experiment as a **regression**.

Use experiments to:

- **Track changes.** Run an experiment for each commit or model and compare the scores over time.
- **Compare models.** Run the same dataset on other models and compare security and cost.
- **Set merge requirements.** Require the scores to pass before a pull request can merge.
- **Inspect a run.** Open a result to see the trace of that run: each model call, each tool call, and the step where the attack succeeded or was stopped.

If you decide to accept the risk that a test case shows, set its status to **Risk accepted**. The test case then has a neutral score, and Relyance records who accepted the risk.

## Benchmarks

The **Overview** tab shows two kinds of results. The **Relyance Security Index** evaluates your agent in its sandbox. Public benchmarks evaluate the model in their own environments, and Relyance shows them for reference with a confidence interval. The first measures the agent. The second measures only the model.

## Reports

| Report | Content |
|---|---|
| **Executive summary** | Security Index, release gate result, and top findings |
| **Technical report** | Each finding with evidence and mitigation |
| **OWASP LLM Top 10** | Test coverage and findings for each risk |
| **Compliance evidence** | How the evaluation supports adversarial testing requirements in the EU AI Act, ISO/IEC 42001, and the NIST AI RMF |
| **Runs (CSV)** | Each test on each configuration |
