# Deploy AIDR Container on AWS These installation steps show you how to install locally into a Minikube cluster. Instructions are also valid for Kubernetes provided by Docker Desktop and a full-fledged Kubernetes cluster. Running locally with Minikube introduces extra latency due to the prompt injection classification model. Latency is not present in a full-fledged Kubernetes cluster. Before deploying Model Scanner, make sure you understand: - [Resource Requirements](/docs/products/aidr-g/resource_requirements) - [Hybrid and Disconnected modes](/docs/products/aidr-g/hybrid_disconnected) - [Proxy and Prompt Analyzer](/docs/products/aidr-g/modes_of_operation) Select OS Select your operating system to view installation instructions. macOS 1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. - For Quay credentials and other requirements, see [Resource Requirements](/docs/products/aidr-g/resource_requirements). ``` cat << EOF > env HIDDENLAYER_USERNAME=%QUAY_USERNAME% HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% EOF ``` Special characters will require wrapping the value in a string. 2. Load the environment variables into your shell. ``` source ./env ``` Select Deployment Type Select your deployment type to view instructions. Hybrid A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with your HiddenLayer API key and secret. - Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` Policy Configuration Environment Variables For other policy configuration environment variables, see [AIDR Configuration](/docs/products/aidr-g/configuration/overview). 1. Make sure Docker Desktop is running. 2. Run the following command to login to the HiddenLayer Quay repository. ``` docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io ``` 3. Run the AIDR container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` Windows 1. Create an environment variable file with Quay credentials. Manually create an `env.bat` file with the following content. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. ``` @echo off set HIDDENLAYER_USERNAME=%QUAY_USERNAME% set HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% ``` Special characters will require wrapping the value in a string. 2. Open a command prompt and load the environment variables into your shell. ``` env ``` Select Deployment Type Select your deployment type to view instructions. Hybrid A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`, then copy and paste the following example into the file. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with your HiddenLayer API key and secret. - Replace `` and `` with information for your AWS instance. ```yaml namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled ``` Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`, then copy and paste the following example into the file. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with information for your AWS instance. ```yaml namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled ``` Policy Configuration Environment Variables For other policy configuration environment variables, see [AIDR Configuration](/docs/products/aidr-g/configuration/overview). 1. Make sure Docker Desktop and the Kubernetes cluster are running. 2. Run the following commands to login to the HiddenLayer Quay repository. ``` docker login --username=%HIDDENLAYER_USERNAME% --password=%HIDDENLAYER_PASSWORD% quay.io ``` 3. Run the AIDR container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` Ubuntu 1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. - For Quay credentials and other requirements, see [Resource Requirements](/docs/products/aidr-g/resource_requirements). ``` cat << EOF > env HIDDENLAYER_USERNAME=%QUAY_USERNAME% HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% EOF ``` Special characters will require wrapping the value in a string. 2. Load the environment variables into your shell. ``` source ./env ``` Select Deployment Type Select your deployment type to view instructions. Hybrid A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with your HiddenLayer API key and secret. - Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See [Hybrid and Disconnected Deployments](/docs/products/aidr-g/hybrid_disconnected) for information about each deployment type. 1. Create a file named `.env.local`. - Use the comments in the example to find configurations for your deployment needs. - File changes to note: - For the EU region, uncomment `hl_region=eu`. - Replace `` with your AIDR license. - Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AISec Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` Policy Configuration Environment Variables For other policy configuration environment variables, see [AIDR Configuration](/docs/products/aidr-g/configuration/overview). 1. Make sure Docker Desktop and the Kubernetes cluster are running. 2. Run the following command to login to the HiddenLayer Quay repository. ``` docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io ``` 3. Run the AIDR container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` ## Health Check To check that the proxy is up and running, open a new terminal and use the following command to ping the health monitor. ``` curl http://localhost:8000/health ``` ## Generate Manifest To write the generated manifest to a file (`manifest.yml`), open a terminal and run the following command. - You must run the AI Detection & Response installer before generating a manifest. - Change `latest` to the AI Detection & Response version that you use. ``` docker run --env-file ./env -v pwd/config:/src/config quay.io/hiddenlayer/distro-enterprise-aidr-genai-installer:latest > manifest.yml ```