Get Started with Agentic Runtime Security

Make your first evaluation with Agentic Runtime Security. Install the SDK, call an evaluation endpoint, and read the result.

This page demonstrates the HiddenLayer Python SDK. The raw API is documented on the Developer Portal (requires a login).

Prerequisites

Access to Agentic Runtime Security enabled for your tenant. If features below are missing in your Console, request access by contacting your HiddenLayer Account Representative or Support (see Overview).
A HiddenLayer client ID and client secret (OAuth2 client credentials), used to generate an access token.
Python 3.9+.
The HiddenLayer SDK: pip install hiddenlayer-sdk.

The evaluation endpoints

Agentic Runtime Security uses three v2 evaluation endpoints, each for a different purpose:

POST /detection/v2/request-evaluations: evaluate a model request (input), inline.
POST /detection/v2/response-evaluations: evaluate a model response (output), inline.
POST /detection/v2/interaction-evaluations: evaluate a set of messages you submit together (a single message up to a full multi-turn exchange) and get a structured result.

In the Python SDK these are client.runtime.evaluate_request(...), client.runtime.evaluate_response(...), and client.runtime.evaluate_interaction(...). This page follows the inline request path; see Evaluation Endpoints for how the endpoints differ and which payload shapes each one accepts. You can make the same evaluation call from an AI gateway or a framework guardrail instead of inline in your application; see Integrations.

Make your first evaluation

Create a Python file. Initialize the client, then evaluate a request with a provider-native payload (an OpenAI Chat Completions request).

from hiddenlayer import HiddenLayer

client = HiddenLayer(
    # By default, the client connects to US Production; to use EU Production:
    # environment="prod-eu",
    # It is recommended to source the authentication credentials from the environment:
    #  - HIDDENLAYER_CLIENT_ID=<client_id>
    #  - HIDDENLAYER_CLIENT_SECRET=<client_secret>
    # They can also be specified directly via:
    # client_id="...",
    # client_secret="...",
    # Alternatively, a bearer token can be sourced from HIDDENLAYER_TOKEN,
    # or specified directly via bearer_token="...".
)

response = client.runtime.evaluate_request(
    # `body` is your provider-native request payload, passed through as-is.
    body={
        "model": "gpt-4o",
        "messages": [
            {"role": "user", "content": "What is the largest moon of Jupiter?"}
        ],
    },
    # Optional headers:
    # hl_project_id="...",          # apply a specific project's policy
    # hl_runtime_session_id="...",  # correlate calls into one session
)
print(response)

Region-specificAuthentication and evaluation are region-specific. Set environment="prod-eu" if you are in the EU, or leave the default (prod-us) for the US. To run against a locally hosted Runtime Security container, set base_url (for example http://localhost:8000).

Reading the result

evaluate_request and evaluate_response return a provider-shaped payload that you forward inline:

If the policy allows or redacts, you get back the request payload (possibly modified, for example with sensitive content redacted) in the provider’s request format. Forward this returned payload to the model in place of your original.
If the policy blocks, you get back a canned block message in the provider’s response format. Return this to the user instead of calling the model. A block is signaled by the hl-runtime-action: BLOCK response header (the call still returns HTTP 200), so check that header to decide whether to enforce — see the inline request and response flow for how to read it. Ignoring it means blocked turns are not enforced.

The request and response endpoints are the simplest inline path: send the provider payload you already have, and forward the one you get back. Use client.runtime.evaluate_interaction(...) instead when you need more than that:

the actual verdict (which rules fired, the threat level, and per-message findings) to log or act on;
support for traffic that isn’t one of the supported provider formats, sent as an explicitly described interaction; or
evaluating out of band, outside the live request path: monitoring, replay, batch analysis, or an integration submitting a captured turn, rather than enforcing inline.

See Evaluation Endpoints for its response shape and examples, and Policy for how detections and actions are defined.

For the full request-and-response enforcement flow, see Evaluation Endpoints.

Set a project

In the Console, a policy is attached to a project. You choose which policy governs a call by telling HiddenLayer which project it belongs to: pass the project ID with the hl_project_id keyword argument, which sets the HL-Project-Id header.

response = client.runtime.evaluate_request(
    body={
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "What is the largest moon of Jupiter?"}],
    },
    hl_project_id="ca87b009-90bd-4724-91c2-f23326acd51a",
)

HiddenLayer evaluates the call against that project’s policy and records the result under the project. See Policy for how policies are built and associated with a project.

Evaluation Endpoints

See how the evaluation endpoints differ and every supported payload shape.

​Prerequisites

​The evaluation endpoints

​Make your first evaluation

​Reading the result

​Set a project

​Next

Evaluation Endpoints

Prerequisites

The evaluation endpoints

Make your first evaluation

Reading the result

Set a project

Next