> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hiddenlayer.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure API Management Integration

This guide walks through inserting HiddenLayer AI Runtime Security into an Azure API Management (APIM) pipeline to scan and act on both user inputs and model outputs (including blocking unsafe content and redacting PII) before they reach the LLM or are returned to the user.

<Note>
  **CLI recommended for most deployments**

  For repeatable APIM setup, use the [`hiddenlayer-apim` CLI](/docs/integrations/azure_apim/cli_overview). The CLI deploys HiddenLayer APIM policy fragments, applies them to APIs, and preserves unrelated APIM policy rules. Use this manual policy guide when you need to inspect or customize the underlying XML directly.
</Note>

Use this manual guide if you cannot run the CLI in your environment, need to understand the XML that APIM executes, or want to build a custom policy by hand.

<Note>
  **Scope of this manual policy guide**

  The XML in this guide is OpenAI / Azure OpenAI chat-completions shaped. It parses OpenAI request and response fields directly and calls HiddenLayer's `/detection/v1/interactions` endpoint. For providers that are not Azure OpenAI (Anthropic, Bedrock, Vertex, etc.), use the CLI's `v2-request-evals` and `v2-response-evals` packages instead — they pass the original provider payload to HiddenLayer's `/detection/v2/request-evaluations` and `/detection/v2/response-evaluations` endpoints and rely on HiddenLayer for provider parsing. See [Fragment packages](/docs/integrations/azure_apim/cli_fragment_packages#provider-compatibility) for per-package provider support.
</Note>

## Prerequisites

### Azure

* An Azure APIM instance.
* An API in that APIM instance that proxies an Azure OpenAI resource and exposes `/chat/completions` and/or `/responses`. This manual guide assumes OpenAI-shaped request and response bodies because the policy below reads OpenAI-specific fields such as `messages`, `input`, and `choices[].message`.

### HiddenLayer

* Access to a HiddenLayer SaaS or containerized (hybrid) instance
* A policy configured with the following:
  * Prompt Injection: **Block**
  * Code: **Block** (output at minimum; input optional)
  * PII: **Redact**
* A project with that policy attached

<Note>
  **Note on PII redaction**

  This guide demonstrates redaction for custom PII entities, which can only be added to a policy via API. If you haven't done that yet, standard entity redaction (e.g., `<PHONE_NUMBER>`) will still work. See [Runtime Security Policy](/docs/products/runtime/policy) to learn how to configure custom entities.
</Note>

## Background: How APIM Policy Works

Azure APIM policy is written in a combination of XML and C#. Full documentation on policy options is available in the [Microsoft APIM policy reference](https://learn.microsoft.com/en-us/azure/api-management/api-management-policies).

To edit policy from the Azure portal, navigate to the operation you want to moderate, then click the angle-bracket icon (`</>`) in either the **Inbound Processing** or **Outbound Processing** pane.

Policy is organized into four sections:

* `<inbound>`: Runs before the request is forwarded to the backend (LLM). Use this to scan user inputs.
* `<backend>`: Controls how requests are forwarded.
* `<outbound>`: Runs after the backend responds. Use this to scan model outputs.
* `<on-error>`: Handles exceptions.

## Complete Policy

The policy example below obtains a SaaS auth token, scans input with HiddenLayer in `<inbound>` (including optional block/redact), forwards the request to Azure OpenAI, then scans the model output in `<outbound>`. It supports both `/chat/completions`-style bodies (with system-role messages filtered out before scanning) and `/responses`-style bodies.

The full policy can be copied into your APIM instance and assumes **SaaS** detection (`https://api.hiddenlayer.ai`). Replace placeholder **named values** (`{{hiddenlayer-client-id}}`, `{{hiddenlayer-client-secret}}`, `{{hiddenlayer-project-id}}`, `{{aoai-api-key}}`) before deploying. You can define these as named variables in Azure API Management and reference them with `{{ }}` as shown.

The sections that follow explain each piece in detail.

```xml theme={null}
<!--
    HiddenLayer APIM Policy — Interactions SaaS Integration
-->
<policies>
    <inbound>
        <base />
        <!-- ========================================== -->
        <!-- Timing                                     -->
        <!-- ========================================== -->
        <set-variable name="request_start" value="@(DateTime.UtcNow)" />
        <!-- ========================================== -->
        <!-- OAuth 2.0 Client Credentials — with cache  -->
        <!-- ========================================== -->
        <cache-lookup-value key="hl-access-token" variable-name="hl_cached_token" />
        <choose>
            <when condition="@(!context.Variables.ContainsKey(&quot;hl_cached_token&quot;))">
                <send-request mode="new" response-variable-name="hl_token_response" timeout="20" ignore-error="false">
                    <set-url>https://api.us.hiddenlayer.ai/oauth2/token</set-url>
                    <set-method>POST</set-method>
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/x-www-form-urlencoded</value>
                    </set-header>
                    <set-body>@{
                        string enc(string s) => System.Net.WebUtility.UrlEncode(s);
                        var clientId     = "{{hiddenlayer-client-id}}";
                        var clientSecret = "{{hiddenlayer-client-secret}}";
                        return $"grant_type=client_credentials&client_id={enc(clientId)}&client_secret={enc(clientSecret)}&audience=https://us.auth.hiddenlayer.ai/";
                    }</set-body>
                </send-request>
                <set-variable name="hl_token_json" value="@(((IResponse)context.Variables[&quot;hl_token_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="hl_access_token" value="@((string)((JObject)context.Variables[&quot;hl_token_json&quot;])[&quot;access_token&quot;])" />
                <set-variable name="hl_expires_in" value="@((int?)((JObject)context.Variables[&quot;hl_token_json&quot;])[&quot;expires_in&quot;])" />
                <cache-store-value key="hl-access-token" value="@((string)context.Variables[&quot;hl_access_token&quot;])" duration="@((int)context.Variables[&quot;hl_expires_in&quot;] - 60)" />
            </when>
            <otherwise>
                <set-variable name="hl_access_token" value="@((string)context.Variables[&quot;hl_cached_token&quot;])" />
            </otherwise>
        </choose>
        <!-- ========================================== -->
        <!-- Extract Request Metadata                   -->
        <!-- ========================================== -->
        <!-- correlation_id ties APIM trace, HL input event, and HL output event together -->
        <set-variable name="correlation_id" value="@(((System.Guid)context.RequestId).ToString())" />
        <!-- requester_id: caller-supplied grouping key, falls back to client IP -->
        <set-variable name="requester_id" value="@{
            string xff = context.Request.Headers.GetValueOrDefault(&quot;X-Forwarded-For&quot;, &quot;&quot;);
            if (!string.IsNullOrEmpty(xff)) { return xff.Split(',')[0].Trim(); }
            return context.Request.IpAddress;
        }" />
        <!-- Preserve full request body for reuse in inbound redact and outbound scan -->
        <set-variable name="original_body" value="@(context.Request.Body.As&lt;JObject&gt;(preserveContent: true))" />
        <!-- Extract model name from request body for HL metadata -->
        <set-variable name="model" value="@(((JObject)context.Variables[&quot;original_body&quot;])[&quot;model&quot;]?.ToString() ?? context.Api.Name)" />
        <!--
            Capture messages array with system role filtered out at source.
            This ensures neither the input scan nor the output scan ever forwards
            the system prompt to the HiddenLayer API.
        -->
        <set-variable name="messages" value="@{
            var all = (JArray)((JObject)context.Variables[&quot;original_body&quot;])[&quot;messages&quot;];
            if (all == null) { return null; }
            var filtered = new JArray();
            foreach (var msg in all) {
                var m = msg as JObject;
                if (m == null) { continue; }
                var role = (string)m[&quot;role&quot;];
                if (!string.IsNullOrEmpty(role) && role.Equals(&quot;system&quot;, StringComparison.OrdinalIgnoreCase)) { continue; }
                filtered.Add(msg);
            }
            return filtered;
        }" />
        <!-- ========================================== -->
        <!-- INPUT SCAN                                 -->
        <!-- ========================================== -->
        <set-variable name="hl_input_start" value="@(DateTime.UtcNow)" />
        <send-request mode="new" response-variable-name="hl_input_response" timeout="10" ignore-error="true">
            <set-url>https://api.hiddenlayer.ai/detection/v1/interactions</set-url>
            <set-method>POST</set-method>
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="Authorization" exists-action="override">
                <value>@("Bearer " + (string)context.Variables["hl_access_token"])</value>
            </set-header>
            <set-header name="HL-Project-Id" exists-action="override">
                <value>{{hiddenlayer-project-id}}</value>
            </set-header>
            <set-header name="X-Correlation-ID" exists-action="override">
                <value>@((string)context.Variables["correlation_id"])</value>
            </set-header>
            <set-header name="X-Requester-Id" exists-action="override">
                <value>@((string)context.Variables["requester_id"])</value>
            </set-header>
            <set-body>@{
                string inputText = null;
                var original = (JObject)context.Variables["original_body"];

                // /chat/completions: extract last user message from filtered messages array
                // (system messages already excluded from context.Variables["messages"])
                var messages = (JArray)context.Variables["messages"];
                if (messages != null) {
                    for (int i = messages.Count - 1; i >= 0; i--) {
                        var m = messages[i] as JObject;
                        if (m == null) { continue; }
                        var role = (string)m["role"];
                        if (!string.IsNullOrEmpty(role) && role.Equals("user", StringComparison.OrdinalIgnoreCase)) {
                            var tok = m["content"];
                            if (tok != null) {
                                inputText = tok.Type == JTokenType.String
                                    ? (string)tok
                                    : tok.ToString(Newtonsoft.Json.Formatting.None);
                            }
                            break;
                        }
                    }
                }
                // /responses endpoint: input field
                else {
                    var input = original?["input"];
                    if (input?.Type == JTokenType.String) {
                        inputText = (string)input;
                    }
                    else if (input?.Type == JTokenType.Array) {
                        var arr = input as JArray;
                        for (int i = arr.Count - 1; i >= 0; i--) {
                            var item = arr[i] as JObject;
                            if ((string)item?["role"] == "user") {
                                inputText = (string)item?["content"];
                                break;
                            }
                        }
                    }
                }

                var payload = new JObject {
                    ["metadata"] = new JObject {
                        ["model"]        = (string)context.Variables["model"],
                        ["requester_id"] = (string)context.Variables["requester_id"],
                        ["provider"]     = "azure openai"
                    },
                    ["input"] = new JObject {
                        ["messages"] = new JArray()
                    }
                };

                if (!string.IsNullOrEmpty(inputText)) {
                    ((JArray)payload["input"]["messages"]).Add(
                        new JObject { ["role"] = "user", ["content"] = inputText }
                    );
                }

                return payload.ToString(Newtonsoft.Json.Formatting.None);
            }</set-body>
        </send-request>
        <set-variable name="hl_input_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;hl_input_start&quot;]).TotalMilliseconds)" />
        <!-- Process input scan result -->
        <choose>
            <when condition="@(
                context.Variables.ContainsKey(&quot;hl_input_response&quot;) &amp;&amp;
                context.Variables[&quot;hl_input_response&quot;] != null &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode >= 200 &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode &lt; 300)">
                <set-variable name="hl_input_result" value="@(((IResponse)context.Variables[&quot;hl_input_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="input_action" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;evaluation.action&quot;)?.ToString() ?? &quot;Allow&quot;)" />
                <set-variable name="input_event_id" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;metadata.event_id&quot;)?.ToString())" />
                <choose>
                    <!-- BLOCK -->
                    <when condition="@(((string)context.Variables[&quot;input_action&quot;]).Equals(&quot;Block&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Inbound-SaaS" severity="error">
                            <message>HiddenLayer blocked inbound request</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;input_event_id&quot;])" />
                            <metadata name="threat_level" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;evaluation.threat_level&quot;)?.ToString())" />
                        </trace>
                        <return-response>
                            <set-status code="403" reason="Blocked by HiddenLayer Policy" />
                            <set-header name="Content-Type" exists-action="override">
                                <value>application/json</value>
                            </set-header>
                            <set-body>@{
                                var result = (JObject)context.Variables["hl_input_result"];
                                var err = new JObject {
                                    ["error"] = new JObject {
                                        ["message"] = "Request blocked by HiddenLayer security policy",
                                        ["type"]    = "blocked_by_policy",
                                        ["code"]    = "content_policy_violation",
                                        ["hiddenlayer"] = new JObject {
                                            ["correlation_id"] = (string)context.Variables["correlation_id"],
                                            ["event_id"]       = (string)context.Variables["input_event_id"],
                                            ["phase"]          = "input",
                                            ["threat_level"]   = result?.SelectToken("evaluation.threat_level")?.ToString(),
                                            ["details"]        = result?.SelectToken("analysis")
                                        }
                                    }
                                };
                                return err.ToString(Newtonsoft.Json.Formatting.None);
                            }</set-body>
                        </return-response>
                    </when>
                    <!-- REDACT -->
                    <when condition="@(((string)context.Variables[&quot;input_action&quot;]).Equals(&quot;Redact&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Inbound-SaaS" severity="information">
                            <message>HiddenLayer redacted inbound request</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;input_event_id&quot;])" />
                        </trace>
                        <set-body>@{
                            var body     = (JObject)context.Variables["original_body"];
                            var result   = (JObject)context.Variables["hl_input_result"];
                            var redacted = (string)result?.SelectToken("modified_data.input.messages[0].content") ?? "";

                            // Rewrite last user message in messages array
                            var msgs = body["messages"] as JArray;
                            if (msgs != null) {
                                for (int i = msgs.Count - 1; i >= 0; i--) {
                                    var m = msgs[i] as JObject;
                                    if (m != null && string.Equals((string)m["role"], "user", StringComparison.OrdinalIgnoreCase)) {
                                        m["content"] = redacted;
                                        break;
                                    }
                                }
                            }
                            // /responses format fallback
                            else if (body["input"] != null) {
                                if (body["input"].Type == JTokenType.String) {
                                    body["input"] = redacted;
                                }
                                else if (body["input"] is JArray arr) {
                                    for (int i = arr.Count - 1; i >= 0; i--) {
                                        var item = arr[i] as JObject;
                                        if ((string)item?["role"] == "user") { item["content"] = redacted; break; }
                                    }
                                }
                            }

                            return body.ToString(Newtonsoft.Json.Formatting.None);
                        }</set-body>
                    </when>
                    <!-- ALLOW: forward unchanged -->
                    <otherwise />
                </choose>
            </when>
            <!-- HL error / timeout: fail-open -->
            <otherwise>
                <trace source="HiddenLayer-Inbound-SaaS" severity="error">
                    <message>HiddenLayer input scan failed or timed out — proceeding fail-open</message>
                    <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                    <metadata name="status_code" value="@(context.Variables.ContainsKey(&quot;hl_input_response&quot;) &amp;&amp; context.Variables[&quot;hl_input_response&quot;] != null ? ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode.ToString() : &quot;null&quot;)" />
                </trace>
            </otherwise>
        </choose>
        <!-- Azure OpenAI auth — set after HL check to avoid sending to HiddenLayer -->
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + "{{aoai-api-key}}")</value>
        </set-header>
        <!-- Strip APIM/HL headers before forwarding to backend -->
        <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
        <set-header name="X-Requester-Id" exists-action="delete" />
        <set-variable name="backend_start" value="@(DateTime.UtcNow)" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
        <set-variable name="backend_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;backend_start&quot;]).TotalMilliseconds)" />
        <!-- Preserve full OpenAI response for redact rewrite if needed -->
        <set-variable name="openai_response" value="@(context.Response.Body.As&lt;JObject&gt;(preserveContent: true))" />
        <set-variable name="assistant_message" value="@(((JObject)context.Variables[&quot;openai_response&quot;])?.SelectToken(&quot;choices[0].message&quot;))" />
        <!-- ========================================== -->
        <!-- OUTPUT SCAN                                -->
        <!-- Only the latest assistant message is sent  -->
        <!-- to HiddenLayer — no history, no system     -->
        <!-- prompt, no extra OpenAI fields.            -->
        <!-- ========================================== -->
        <set-variable name="hl_output_start" value="@(DateTime.UtcNow)" />
        <send-request mode="new" response-variable-name="hl_output_response" timeout="10" ignore-error="true">
            <set-url>https://api.hiddenlayer.ai/detection/v1/interactions</set-url>
            <set-method>POST</set-method>
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="Authorization" exists-action="override">
                <value>@("Bearer " + (string)context.Variables["hl_access_token"])</value>
            </set-header>
            <set-header name="HL-Project-Id" exists-action="override">
                <value>{{hiddenlayer-project-id}}</value>
            </set-header>
            <set-header name="X-Correlation-ID" exists-action="override">
                <value>@((string)context.Variables["correlation_id"])</value>
            </set-header>
            <set-header name="X-Requester-Id" exists-action="override">
                <value>@(context.Variables.ContainsKey("requester_id") ? (string)context.Variables["requester_id"] : context.Request.IpAddress)</value>
            </set-header>
            <set-body>@{
                // Send only the latest assistant message, normalized to {role, content}.
                // Strips extra OpenAI fields (tool_calls, refusal, function_call, etc.)
                // and excludes all conversation history and system prompt.
                var assistantMessage = context.Variables["assistant_message"] as JObject;

                var outputMessages = new JArray();
                if (assistantMessage != null) {
                    outputMessages.Add(new JObject {
                        ["role"]    = "assistant",
                        ["content"] = assistantMessage["content"]?.ToString() ?? ""
                    });
                }

                var payload = new JObject {
                    ["metadata"] = new JObject {
                        ["model"]        = (string)context.Variables["model"],
                        ["requester_id"] = context.Variables.ContainsKey("requester_id")
                                               ? (string)context.Variables["requester_id"]
                                               : context.Request.IpAddress,
                        ["provider"]     = "azure openai"
                    },
                    ["output"] = new JObject {
                        ["messages"] = outputMessages
                    }
                };

                return payload.ToString(Newtonsoft.Json.Formatting.None);
            }</set-body>
        </send-request>
        <set-variable name="hl_output_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;hl_output_start&quot;]).TotalMilliseconds)" />
        <!-- Process output scan result -->
        <choose>
            <when condition="@(
                context.Variables.ContainsKey(&quot;hl_output_response&quot;) &amp;&amp;
                context.Variables[&quot;hl_output_response&quot;] != null &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode >= 200 &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode &lt; 300)">
                <set-variable name="hl_output_result" value="@(((IResponse)context.Variables[&quot;hl_output_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="output_action" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;evaluation.action&quot;)?.ToString() ?? &quot;Allow&quot;)" />
                <set-variable name="output_event_id" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;metadata.event_id&quot;)?.ToString())" />
                <choose>
                    <!-- BLOCK -->
                    <when condition="@(((string)context.Variables[&quot;output_action&quot;]).Equals(&quot;Block&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Outbound-SaaS" severity="error">
                            <message>HiddenLayer blocked outbound response</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;output_event_id&quot;])" />
                            <metadata name="threat_level" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;evaluation.threat_level&quot;)?.ToString())" />
                        </trace>
                        <return-response>
                            <set-status code="403" reason="Response Blocked by HiddenLayer Policy" />
                            <set-header name="Content-Type" exists-action="override">
                                <value>application/json</value>
                            </set-header>
                            <set-body>@{
                                var result = (JObject)context.Variables["hl_output_result"];
                                var err = new JObject {
                                    ["error"] = new JObject {
                                        ["message"] = "Response blocked by HiddenLayer security policy",
                                        ["type"]    = "response_blocked_by_policy",
                                        ["code"]    = "content_policy_violation",
                                        ["hiddenlayer"] = new JObject {
                                            ["correlation_id"] = (string)context.Variables["correlation_id"],
                                            ["event_id"]       = (string)context.Variables["output_event_id"],
                                            ["phase"]          = "output",
                                            ["threat_level"]   = result?.SelectToken("evaluation.threat_level")?.ToString(),
                                            ["details"]        = result?.SelectToken("analysis")
                                        }
                                    }
                                };
                                return err.ToString(Newtonsoft.Json.Formatting.None);
                            }</set-body>
                        </return-response>
                    </when>
                    <!-- REDACT -->
                    <when condition="@(((string)context.Variables[&quot;output_action&quot;]).Equals(&quot;Redact&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Outbound-SaaS" severity="information">
                            <message>HiddenLayer redacted outbound response</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;output_event_id&quot;])" />
                        </trace>
                        <set-body>@{
                            var response = (JObject)context.Variables["openai_response"];
                            var result   = (JObject)context.Variables["hl_output_result"];
                            // modified_data.output.messages contains the redacted assistant turn
                            var redactedMsgs = result?.SelectToken("modified_data.output.messages") as JArray;
                            if (redactedMsgs != null && redactedMsgs.Count > 0) {
                                var safeMsg = redactedMsgs[redactedMsgs.Count - 1] as JObject;
                                var choices = response?["choices"] as JArray;
                                if (choices != null && choices.Count > 0) {
                                    var firstChoice = choices[0] as JObject;
                                    if (firstChoice != null && safeMsg != null) {
                                        firstChoice["message"] = safeMsg;
                                    }
                                }
                            }
                            return response.ToString(Newtonsoft.Json.Formatting.None);
                        }</set-body>
                    </when>
                    <!-- ALLOW: return response unchanged -->
                    <otherwise />
                </choose>
            </when>
            <!-- HL error / timeout: fail-open -->
            <otherwise>
                <trace source="HiddenLayer-Outbound-SaaS" severity="error">
                    <message>HiddenLayer output scan failed or timed out — returning original response fail-open</message>
                    <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                    <metadata name="status_code" value="@(context.Variables.ContainsKey(&quot;hl_output_response&quot;) &amp;&amp; context.Variables[&quot;hl_output_response&quot;] != null ? ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode.ToString() : &quot;null&quot;)" />
                </trace>
            </otherwise>
        </choose>
        <!-- ========================================== -->
        <!-- Timing Headers                             -->
        <!-- ========================================== -->
        <set-variable name="total_duration_ms" value="@(context.Elapsed.TotalMilliseconds)" />
        <set-variable name="hl_total_ms" value="@((double)context.Variables[&quot;hl_input_duration_ms&quot;] + (double)context.Variables[&quot;hl_output_duration_ms&quot;])" />
        <set-variable name="apim_overhead_ms" value="@((double)context.Variables[&quot;total_duration_ms&quot;] - (double)context.Variables[&quot;hl_total_ms&quot;] - (double)context.Variables[&quot;backend_duration_ms&quot;])" />
        <trace source="HiddenLayer-Timing" severity="information">
            <message>Request timing summary</message>
            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
            <metadata name="total_ms" value="@(((double)context.Variables[&quot;total_duration_ms&quot;]).ToString())" />
            <metadata name="hl_input_ms" value="@(((double)context.Variables[&quot;hl_input_duration_ms&quot;]).ToString())" />
            <metadata name="hl_output_ms" value="@(((double)context.Variables[&quot;hl_output_duration_ms&quot;]).ToString())" />
            <metadata name="hl_total_ms" value="@(((double)context.Variables[&quot;hl_total_ms&quot;]).ToString())" />
            <metadata name="openai_ms" value="@(((double)context.Variables[&quot;backend_duration_ms&quot;]).ToString())" />
            <metadata name="apim_overhead_ms" value="@(((double)context.Variables[&quot;apim_overhead_ms&quot;]).ToString())" />
            <metadata name="status_code" value="@(context.Response.StatusCode.ToString())" />
        </trace>
        <set-header name="X-Correlation-ID" exists-action="override">
            <value>@((string)context.Variables["correlation_id"])</value>
        </set-header>
        <set-header name="X-Total-Duration-Ms" exists-action="override">
            <value>@(((double)context.Variables["total_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-HiddenLayer-Input-Ms" exists-action="override">
            <value>@(((double)context.Variables["hl_input_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-HiddenLayer-Output-Ms" exists-action="override">
            <value>@(((double)context.Variables["hl_output_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-OpenAI-Ms" exists-action="override">
            <value>@(((double)context.Variables["backend_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-APIM-Overhead-Ms" exists-action="override">
            <value>@(((double)context.Variables["apim_overhead_ms"]).ToString())</value>
        </set-header>
    </outbound>
    <on-error>
        <base />
        <trace source="HiddenLayer-Policy-Error" severity="error">
            <message>@("Unhandled policy error: " + (context.LastError?.Message ?? "unknown"))</message>
            <metadata name="correlation_id" value="@(context.Variables.ContainsKey(&quot;correlation_id&quot;) ? (string)context.Variables[&quot;correlation_id&quot;] : &quot;not-set&quot;)" />
        </trace>
    </on-error>
</policies>
```

## How It Works: Section by Section

### Calling HiddenLayer Inbound (User Input)

The inbound section intercepts the user's request before it reaches the LLM and POSTs to HiddenLayer's `/detection/v1/interactions` endpoint.

For **`/chat/completions`** requests, the policy builds a filtered copy of `messages` that omits `system` role entries (so the system prompt is never sent to HiddenLayer). It scans the latest **user** message content. For **`/responses`**-style bodies, it reads the `input` field (string or message array) instead.

Requests to HiddenLayer include `Authorization` (Bearer token from SaaS OAuth), `HL-Project-Id`, `X-Correlation-ID` (APIM `RequestId`), and `X-Requester-Id` (first hop from `X-Forwarded-For`, else client IP).

### Responding Based on the HiddenLayer Evaluation

The policy reads **`evaluation.action`** from the JSON response (`Block`, `Redact`, or allow).

**If HiddenLayer returns a non-2xx status or times out**, the policy **fails open**: it traces the failure and forwards the original request to Azure OpenAI.

**If the evaluation action is `Block`**, the caller receives **403** with a JSON error body that includes correlation and event metadata.

**If the evaluation action is `Redact`**, the inbound body is rewritten using `modified_data.input.messages[0].content` (for chat completions, the last user message is updated; for `/responses`, the `input` field is updated accordingly).

After the HiddenLayer inbound check completes, the policy sets the Azure OpenAI **`Authorization`** header (`{{aoai-api-key}}`), removes `Ocp-Apim-Subscription-Key` and `X-Requester-Id`, and records timing for the backend call.

### Calling HiddenLayer Outbound (Model Output)

The outbound section reads the assistant turn from **`choices[0].message`** (chat completions shape), sends only that normalized message to `/detection/v1/interactions`, and applies the same Block (**403**) / Redact / fail-open behavior. Redaction merges `modified_data.output.messages` back into `choices[0].message`.

Timing / correlation headers (`X-Correlation-ID`, `X-HiddenLayer-*-Ms`, etc.) are added on the response for observability.

## Appendix

### Correlating HiddenLayer Logs with APIM Logs

The **complete policy above** already stores APIM's `RequestId` as `correlation_id` and sends **`X-Correlation-ID`** on HiddenLayer requests and on the client response. That matches gateway logs without extra steps.

If you are building a smaller policy or fragment by hand, you can add the same pattern explicitly:

```xml theme={null}
<set-variable name="corrId" value="@(((System.Guid)context.RequestId).ToString())" />
```

```xml theme={null}
<set-header name="X-Correlation-ID" exists-action="override">
  <value>@((string)context.Variables["corrId"])</value>
</set-header>
```

### APIM Policy Fragments

Policy fragments are centrally managed, reusable XML code snippets that can enable consistent integration with AI Runtime across the APIM environment. For more information from Microsoft, see [Reuse Policy Configurations in API Management](https://learn.microsoft.com/en-us/azure/api-management/policy-fragments).

It is common to maintain the **inbound** HiddenLayer logic (OAuth, input scan, Azure OpenAI headers) and the **outbound** HiddenLayer logic (output scan, timing headers) as **separate fragments**. That lets you reuse the HiddenLayer pieces across APIs while composing them with unrelated APIM policy (rate limiting, caching, JWT validation, transformation steps, or other backends) in the order your operation requires.

The OAuth token cache logic shown in the inbound section above can be extracted into a fragment (for example, named `hiddenlayer-saas-auth`) and referenced in any policy using:

```xml theme={null}
<include-fragment fragment-id="hiddenlayer-saas-auth" />
```

Example usage in a policy:

```xml theme={null}
<policies>
  <inbound>
    <include-fragment fragment-id="hiddenlayer-saas-auth" />
    [...]
  </inbound>
  [...]
</policies>
```