Deploy AI Runtime Security using Helm

Deploy AI Runtime Security to a Kubernetes cluster using Helm.

• Access to a Kubernetes cluster
• kubectl — Kubernetes command-line tool
• Helm — Kubernetes package manager (v3+)
• Resource Requirements — license keys, tools, and scaling guidance
• Hybrid and Disconnected Modes — connection mode details

Create a values.yaml file to override the chart's default settings. Application configuration goes inside config.settings.yaml as an embedded YAML string block. Secrets (auth credentials and license) go in the env: block.

Choose the resource type for your deployment.

• CPU: Deploy on resources that utilize CPUs only. This is the default image.
• GPU: Deploy on resources that utilize a CUDA-enabled GPU.

aidr_genai:
  image:
    repository: distro-enterprise-aidr-genai
  env:
    HL_LLM_PROXY_CLIENT_ID: "secret:<base64-encoded-client-id>"
    HL_LLM_PROXY_CLIENT_SECRET: "secret:<base64-encoded-client-secret>"
    HL_LICENSE: "secret:<base64-encoded-license-key>"
    HL_LLM_PROXY_TOKEN_URL: "https://auth.hiddenlayer.ai"
    HL_LLM_PROXY_MLDR_CONNECTION_TYPE: "hybrid"
    HL_LLM_PROXY_MLDR_BASE_URL: "https://api.us.hiddenlayer.ai" # US region
    # HL_LLM_PROXY_MLDR_BASE_URL: "https://api.eu.hiddenlayer.ai" # EU region
    HL_LLM_PROXY_LOG_LEVEL: "info"
    HL_LLM_PROXY_MLDR_COLLECT_PROMPT: "true"

1. Run the following command in a terminal to log in to the HiddenLayer registry.

   • The username is your Registry username.
   • The password is your License ID.
   • For more information, see Resource Requirements.
   helm registry login registry.hiddenlayer.ai --username <email specified for registry> --password <License ID>

For users that would like to use their own repository, see the steps below on how to pull the relevant images. Skip these steps if this is not applicable to you.

1. Create a values.yaml file to customize installation.

   • See Create a Helm Values File above.

2. Run the following command to deploy Runtime Security.

   helm upgrade --install aidr-genai oci://registry.hiddenlayer.ai/aidr-genai/stable/aidr-genai \
     --namespace aidr-genai --create-namespace \
     -f values.yaml

1. Check that all pods are running:

   kubectl get pods -n aidr-genai

2. Port-forward the service to your local machine:

   kubectl port-forward svc/aidr-genai 8000:80 -n aidr-genai

3. Verify the health endpoint:

   curl http://localhost:8000/health

Once deployed, you can analyze LLM input and output by sending requests to the Interactions endpoint:

curl -X POST http://<service-endpoint>:8000/detection/v1/interactions \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {
      "model": "gpt-5",
      "requester_id": "user-1234",
      "provider": "openai"
    },
    "input": {
      "messages": [
        {
          "role": "user",
          "content": "What is the largest moon of Jupiter?"
        }
      ]
    },
    "output": {
      "messages": [
        {
          "role": "assistant",
          "content": "The largest moon of Jupiter is Ganymede."
        }
      ]
    }
  }'

For SDK examples and the full response format, see Getting Started with Interactions.

The following sections cover additional configuration beyond the baseline. The examples in Create a Helm Values File include working defaults for all of these — adjust as needed after verifying your deployment.

The chart creates a Horizontal Pod Autoscaler (HPA) when resources.targetUtilization.cpu and resources.requests.cpu are set. Use replicas.min and replicas.max to control scaling bounds.

Allocate 8 CPU per replica and set OMP_NUM_THREADS: 8 to match. Scale replicas to fill node capacity — for example, 4 replicas on a 32-vCPU node. See Resource Requirements for detailed scaling guidance.

aidr_genai:
  resources:
    targetUtilization:
      cpu: 75
    requests:
      cpu: 8
      memory: 4096Mi
    limits:
      memory: 4096Mi
  replicas:
    min: 2
    max: 8

In Hybrid mode, the AI Runtime Security pods make outbound connections to the HiddenLayer Console (auth.hiddenlayer.ai and the regional api.*.hiddenlayer.ai endpoint). If your cluster routes external traffic through a forward proxy, configure it by setting the standard HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables in the aidr_genai.env block of your values.yaml. This section does not apply to Disconnected mode, which sends no outbound traffic to HiddenLayer.

aidr_genai:
  env:
    # ...existing credentials and settings...
    HTTP_PROXY: "http://proxy.internal.example.com:8080"
    HTTPS_PROXY: "http://proxy.internal.example.com:8080"
    # Bypass the proxy for in-cluster and internal traffic
    NO_PROXY: "localhost,127.0.0.1,.svc,.svc.cluster.local,kubernetes.default.svc,<cluster-cidr>"

aidr_genai:
  config:
    ca_bundle.crt: |
      -----BEGIN CERTIFICATE-----
      <your proxy's CA certificate>
      -----END CERTIFICATE-----
  env:
    # ...existing credentials and settings...
    SSL_CERT_FILE: $(CONFIG_PATH)/ca_bundle.crt

Configure a detection policy using environment variables inside the values.yaml file.

What the following example does:

• Uses the AI Runtime Security image for a CPU-only resource.

• Will send results to the HiddenLayer Console (US region).

• For Prompt Injection:

  • Detect high severity prompt injection, but will not block it (alert-only).

• For Personally Identifiable Information (PII):

  • Detect PII in input and output, but will not block it (alert-only).
  • Redact PII in input and output, replacing the redacted content with an entity label (example: [PHONE_NUMBER]).

aidr_genai:
  image:
    repository: distro-enterprise-aidr-genai # CPU image
    # repository: distro-enterprise-aidr-genai-cuda # GPU image
  # resources: # GPU resources
    # limits:
      # nvidia.com/gpu: 1
  env:
    HL_LLM_PROXY_CLIENT_ID: "secret:<base64-encoded-client-id>"
    HL_LLM_PROXY_CLIENT_SECRET: "secret:<base64-encoded-client-secret>"
    HL_LICENSE: "secret:<base64-encoded-license-key>"
    HL_LLM_PROXY_TOKEN_URL: "https://auth.hiddenlayer.ai"
    HL_LLM_PROXY_MLDR_CONNECTION_TYPE: "hybrid"
    HL_LLM_PROXY_MLDR_BASE_URL: "https://api.us.hiddenlayer.ai" # US region
    # HL_LLM_PROXY_MLDR_BASE_URL: "https://api.eu.hiddenlayer.ai" # EU region
    HL_LLM_PROXY_LOG_LEVEL: "info"
    HL_LLM_PROXY_MLDR_COLLECT_PROMPT: "true"
    # Prompt Injection
    HL_LLM_SKIP_PROMPT_INJECTION_DETECTION: "false"            # detection is enabled
    HL_LLM_PROXY_CONVICTION_SEVERITY_PROMPT_INJECTION: "High"  # high severity threshold for detections
    HL_LLM_BLOCK_PROMPT_INJECTION: "false"                     # alert only, do not block
    # PII Detection & Redaction
    HL_LLM_SKIP_INPUT_PII_DETECTION: "false"                   # input PII detection enabled
    HL_LLM_SKIP_OUTPUT_PII_DETECTION: "false"                  # output PII detection enabled
    HL_LLM_BLOCK_INPUT_PII: "false"                            # alert only, do not block input
    HL_LLM_BLOCK_OUTPUT_PII: "false"                           # alert only, do not block output
    HL_LLM_REDACT_INPUT_PII: "true"                            # redact PII in input
    HL_LLM_REDACT_OUTPUT_PII: "true"                           # redact PII in output
    HL_LLM_REDACT_TYPE: "ENTITY"                               # replace with entity label e.g. [PHONE_NUMBER]