> ## Documentation Index > Fetch the complete documentation index at: https://docs.hiddenlayer.ai/llms.txt > Use this file to discover all available pages before exploring further. # Evaluation Endpoints Agentic Runtime Security exposes three evaluation endpoints. This page describes the endpoints you call to evaluate traffic, what each one accepts, and what each returns, along with the provider formats they understand. *** ## The evaluation endpoints Each endpoint serves a distinct job. They differ on two things: the **input shape** you send, and **what you get back**. | Endpoint | SDK method | When called | Accepts | Returns | | -------------------------------------------- | ------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | `POST /detection/v2/request-evaluations` | `client.runtime.evaluate_request` | Before the model runs (inline) | Provider-native **request** shapes (auto-detected) | The (possibly modified) request payload to forward inline, or (if blocked) the response payload to return | | `POST /detection/v2/response-evaluations` | `client.runtime.evaluate_response` | After the model returns (inline) | Provider-native **response** shapes (auto-detected) | The (possibly modified) response payload to forward inline | | `POST /detection/v2/interaction-evaluations` | `client.runtime.evaluate_interaction` | Any time, inline or out of band | An interaction you describe directly, either provider-native or as an explicitly described interaction with messages and typed content parts, plus metadata | A structured outcome (action, threat level, detections) | The **request** and **response** endpoints (`evaluate_request` / `evaluate_response`) are pass-through: you send a provider-native request or response, and you get back a payload in the same provider format to forward in the live serving path. The policy returns the original (possibly redacted) payload when it allows or redacts, or a block message when it blocks. Use them for inline enforcement. An AI gateway or proxy in front of your LLM traffic typically calls these inline endpoints, forwarding the provider-native payloads it already sees. The **interaction** endpoint (`evaluate_interaction`) is different on both axes: you submit an interaction you describe explicitly, an envelope of `{ metadata, interaction }`, and you get back a structured outcome rather than a payload to forward. Use it instead of the inline request and response endpoints when you need: * the actual verdict (which rules fired, the threat level, and per-message findings) to log or act on, rather than just a payload to forward; * support for traffic that isn't one of the [supported provider formats](#supported-provider-formats), sent as an explicitly described interaction; or * to evaluate out of band, outside the live request path: monitoring, replay, batch analysis, or an integration submitting a captured turn, rather than enforcing inline. The call itself always returns its result synchronously, even when you use it out of band. ### request-evaluations Evaluate a raw provider **request** (input), before the model runs (inline, in the request path). It accepts provider-native request shapes only, which are auto-detected. ### response-evaluations Evaluate a raw provider **response** (output), after the model returns. It accepts provider-native response shapes only, auto-detected. Pair a response with its request by sending the same `HL-Roundtrip-Id` on both (see [Agentic Sessions](/docs/products/runtime/agentic/sessions)). ### Inline request and response flow To enforce on a live turn, wrap your model call between a request evaluation and a response evaluation: 1. Call `evaluate_request(...)` with the provider request. 2. If the response carries the header `hl-runtime-action: BLOCK`, the policy blocked the turn: return the returned payload (a provider response-shaped block message) without calling the model. 3. Otherwise forward the returned (possibly redacted) request payload to the model, not your original. 4. Call `evaluate_response(...)` on the model's response. 5. Return the returned (possibly redacted) response payload. A block is signaled by the `hl-runtime-action` response header, not by the body or the HTTP status (both endpoints return `200` even on a block). To read that header you call the endpoint through the SDK's raw-response accessor (`with_raw_response`, or `with_streaming_response`); the plain `client.runtime.evaluate_request(...)` method returns only the parsed body. Pass the correlation ids so the two evaluations reconstruct into one session: one `hl_runtime_session_id` for the whole workflow, and one `HL-Roundtrip-Id` per turn (the same value on the request and its response) via `extra_headers`. See [Agentic Sessions](/docs/products/runtime/agentic/sessions) for where these ids come from. ```python theme={null} import uuid # One session id for the whole workflow (created once, reused on every call), and one # roundtrip id per turn that pairs a request with its response. session_id = str(uuid.uuid4()) # in practice, created once when the workflow begins roundtrip_id = str(uuid.uuid4()) # one per turn # One inline turn: screen the request, call the model, screen the response. raw = client.runtime.with_raw_response.evaluate_request( body=request_payload, hl_runtime_session_id=session_id, extra_headers={"HL-Roundtrip-Id": roundtrip_id}, ) if raw.headers.get("hl-runtime-action") == "BLOCK": # Policy blocked. `raw.parse()` is a provider response-shaped block message # ("Message was blocked"); return it without calling the model. return raw.parse() checked_request = raw.parse() # allowed/redacted request payload — forward this, not your original model_response = call_model(checked_request) # Reuse the SAME roundtrip id so the two halves pair up. resp = client.runtime.with_raw_response.evaluate_response( body=model_response, hl_runtime_session_id=session_id, extra_headers={"HL-Roundtrip-Id": roundtrip_id}, ) if resp.headers.get("hl-runtime-action") == "BLOCK": return resp.parse() # provider response-shaped block message return resp.parse() # the (possibly redacted) payload to return to the caller ``` **Handle blocks explicitly** Both inline endpoints return HTTP `200` whether the policy allows, redacts, or blocks — the outcome is carried only in the `hl-runtime-action` header. If your integration ignores that header and forwards or returns the payload unconditionally, blocked turns are not enforced (a fail-open). Always branch on `hl-runtime-action == "BLOCK"` before calling the model or returning the result, which is why the flow above reads the header through `with_raw_response`. ### interaction-evaluations Use `interaction-evaluations` when you want to submit an interaction directly and get structured results back, rather than a provider-shaped payload to forward inline. This is the endpoint for: * logging or acting on the policy action, threat level, detections, and per-message findings; * evaluating captured traffic out of band, such as monitoring, replay, batch analysis, or an integration submitting a turn; * evaluating traffic that is not one of the provider-native request or response formats accepted by the inline endpoints. Submit the interaction as `{ metadata, interaction }`. The `metadata` object identifies the model, provider, requester, and optional external session id. The `interaction` can be either: * a supported provider-native payload, such as OpenAI or Anthropic; or * an explicitly described interaction: `messages[]` with roles and typed content parts such as `text`, `tool_use`, and `tool_result`. (This is sometimes called the normalized interaction format in API schemas.) The explicitly described interaction format is accepted only by `interaction-evaluations`; the request and response endpoints expect provider-native request or response bodies because they return provider-shaped payloads for inline forwarding. Traffic that isn't one of the [supported provider formats](#supported-provider-formats) can be sent as an explicitly described interaction here. Its response includes: * `outcome.action`: `NONE`, `DETECT`, `REDACT`, or `BLOCK`. * `outcome.threat_level`: `NONE`, `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. * `outcome.detections[]`: each with a `rule_name` and `risk_level`. * `outcome.effective_interaction`: the payload to forward downstream (with any redactions or substitutions applied). * `evaluated_interaction.messages[].analysis.signals`: the per-message signal findings. For the full response schema, see the Developer Portal (requires a login). ## Supported provider formats Agentic Runtime Security accepts common LLM provider wire formats directly, plus an explicitly described interaction format for `interaction-evaluations`. Provider shapes are auto-detected from the JSON body; there is no format field to set. * **OpenAI Chat Completions** (request and response) * **OpenAI Responses API** (request and response) * **Anthropic Messages** (request and response) Request and response are distinct shapes, not one combined object. Send the request shape to the request endpoint and the response shape to the response endpoint. For providers not listed here, send an explicitly described interaction via `interaction-evaluations`. ## Tool calls and agentic content Tool calls are supported across all formats: * OpenAI: `tool_calls` and `role: tool` messages. * Anthropic: `tool_use` and `tool_result` content blocks. * OpenAI Responses: function/tool calls. * Explicitly described interaction: `tool_use` and `tool_result` typed content parts. ## Examples The inline examples pass the correlation ids (`hl_runtime_session_id` and a per-turn `HL-Roundtrip-Id`) so the calls reconstruct into one session. Here `session_id` is created once for the workflow and `roundtrip_id` is generated per turn; see [Agentic Sessions](/docs/products/runtime/agentic/sessions) for details. ### request-evaluations: OpenAI Chat Completions request ```python theme={null} response = client.runtime.evaluate_request( body={ "model": "gpt-4o", "messages": [ {"role": "user", "content": "What is the largest moon of Jupiter?"} ], }, hl_runtime_session_id=session_id, # groups this call into the session extra_headers={"HL-Roundtrip-Id": roundtrip_id}, # pairs this request with its response ) ``` ### response-evaluations: OpenAI Chat Completions response ```python theme={null} response = client.runtime.evaluate_response( body={ "id": "chatcmpl-123", "object": "chat.completion", "model": "gpt-4o", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The largest moon of Jupiter is Ganymede.", }, "finish_reason": "stop", } ], }, hl_runtime_session_id=session_id, # same session id as the request extra_headers={"HL-Roundtrip-Id": roundtrip_id}, # same roundtrip id pairs the two halves ) ``` ### interaction-evaluations: explicitly described interaction An explicitly described interaction uses `messages[]` with typed content parts and no top-level `model`. ```python theme={null} response = client.runtime.evaluate_interaction( metadata={ "model": "gpt-4o", "provider": "openai", "requester_id": "agent-001", "external_session_id": "7f3a1c2e-0a4b-4c8d-9e1f-2b3c4d5e6f70", }, interaction={ "messages": [ { "role": "user", "content": [{"type": "text", "text": "Summarize the latest sales report."}], }, { "role": "assistant", "content": [ { "type": "tool_use", "id": "call_1", "tool_name": "fetch_report", "tool_input": {"name": "sales-q2"}, } ], }, { "role": "tool", "content": [ {"type": "tool_result", "id": "call_1", "result": "Q2 revenue up 12% over Q1."} ], }, { "role": "assistant", "content": [{"type": "text", "text": "Q2 revenue rose 12% versus Q1."}], }, ] }, ) ``` ## Next Correlate an agent's many calls into one replayable session.