Skip to content
AI Detection & Response (...
/
/
Helm

Deploy AIDR Container on AWS using Helm

Deployment for AIDR using a Helm file requires creating a config.yaml file, then downloading and deploying as a container to a Kubernetes cluster.

For prerequisites, including licenses, see Resource Requirements.

Create a Config YAML file

To deploy AIDR, a config.yaml file must be generated with the appropriate settings. Use the following example and modify as needed.

Select Provider

Select your provider to view installation instructions.

aidr_genai:

  platform:
    api-connection:
      base-url: "https://<HiddenLayer Platform Hostname>"
      max-retry-count: 3
      ## Set type: disabled to install AIDR without a connection to the Platform.
      ## If AIDR is deployed and you want to change the type, update this yaml file, then run a helm upgrade command.
      type: "hybrid"
  
  proxy:
    log-level: "info"

    provider:
      timeout-in-seconds: 600

      aws:
        region:  # "us-east-1"
        enable-instance-profile-credentials: false
        credential-provider: "instance"  # container

        sagemaker:
          base-url:  # "https://runtime.sagemaker.{region}.amazonaws.com"

        bedrock:
          base-url:  # "https://bedrock-runtime.{region}.amazonaws.com"

        credentials:
          #- name:
          #  access-key-id:
          #  secret-access-key:
          #  session-token:
          #  region:
          #  sagemaker-base-url:
          #  bedrock-base-url:

Log In to Helm

The following is required to log in to the HiddenLayer helm registry.

  1. Run the following command in a terminal to log in to the HiddenLayer registry.

    • The username is your Registry username.
    • The password is your License ID.
    • For more information, see Resource Requirements.
    helm registry login registry.hiddenlayer.ai --username <email specified for registry> --password <License ID>

Install

Use the following instructions to install AIDR.

  1. Create a config.yaml file to customize installation

    • See above for an example config.yaml file.

  2. Run the following command to deploy AIDR.

    helm install oci://registry.hiddenlayer.ai/aidr-genai/stable/distro-enterprise-aidr-genai-installer --version 2.1.0 -f config.yaml

Known Issue

There is a known issue for where provider credentials for AWS are not utilized when using the config.yaml file.

  • Environment variables cannot be used with the installer to set AWS credentials.

  • Work around: Add an Authorization header in the request to the LLM Proxy.

    #Example
Authorization: Credential=AWSAccessKeyId
    #Example
curl -XPOST http://localhost:8000/api/v1/proxy/bedrock/model/meta.llama2-13b-chat-v1 \
  -H "Content-Type: application/json" \
  -H "Authorization: Credential=AWSAccessKeyId" \
  -d '{
    "prompt": "Hello, how are you?",
    "max_tokens_to_sample": 300
  }'
Previous page
Next page