LLM Proxy (1)
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

HiddenLayer LLM-Proxy is a service that provides detection and response capabilities for Generative AI solutions in the HiddenLayer AI Security Platform.

Locally deployed AIDR proxy in Hybrid mode

The LLM Proxy API endpoint is supported only for a locally deployed AIDR proxy running in Hybrid mode. This API is not supported in HiddenLayer SaaS.
The documentation is presented here on the documentation portal to help reduce confusion between a locally running proxy and the SaaS proxy.
For more API documentation, see the HiddenLayer Developer Portal.

Download OpenAPI description

llm_proxy_api.json

llm_proxy_api.yaml

Servers

Mock server

https://docs.hiddenlayer.ai/_mock/docs/products/aidr-g/llm_proxy_api/

Self-hosted server

https://<YOUR-SELF-HOSTED-AIDR-INSTANCE-ENDPOINT>/

Health and Metrics
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Operations

OpenAI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Operations

Azure
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Operations

Proxy Azure endpoint

Request

Generate text or chat completions for a given Azure endpoint and return response and analysis

Path

endpoint_namestringrequired

The name of the Azure endpoint to use

Query

regionstring

The region of the Azure endpoint

Headers

HL-Project-Idstring

The ID or alias for the Project to govern the request processing.

Example: internal-search-chatbot

X-Requester-Idstring

The identifier for the requester to be used if MLDR is enabled

X-LLM-Block-Unsafeboolean

Whether to block unsafe input and output

Default false

X-LLM-Block-Unsafe-Inputboolean

Whether to block unsafe input

Default false

X-LLM-Block-Unsafe-Outputboolean

Whether to block unsafe output

Default false

X-LLM-Skip-Prompt-Injection-Detectionboolean

Whether to skip prompt injection detection

Default false

X-LLM-Block-Prompt-Injectionboolean

Whether to block prompt injection

Default false

X-LLM-Prompt-Injection-Scan-Typestring(HiddenLayerAnalysisPromptInjectionEnum)

The type of prompt injection scan to use

Default quick

Enum"quick""full"

Example: quick

X-LLM-Skip-Input-DOS-Detectionboolean

Whether to skip input denial of service detection

Default false

X-LLM-Block-Input-DOS-Detectionboolean

Whether to block input denial of service detection

Default false

X-LLM-Input-DOS-Detection-Thresholdstring

The threshold for input denial of service detection

Default 4096

X-LLM-Skip-Input-PII-Detectionboolean

Whether to skip input personally identifiable information detection

Default false

X-LLM-Skip-Output-PII-Detectionboolean

Whether to skip output personally identifiable information detection

Default false

X-LLM-Block-Input-PIIboolean

Whether to block input personally identifiable information detection

Default false

X-LLM-Block-Output-PIIboolean

Whether to block output personally identifiable information detection

Default false

X-LLM-Redact-Input-PIIboolean

Whether to redact input personally identifiable information

Default false

X-LLM-Redact-Output-PIIboolean

Whether to redact output personally identifiable information

Default false

X-LLM-Redact-Typestring(HiddenLayerAnalysisRedactTypeEnum)

The type of redaction to use

Default entity

Enum"entity""strict"

Example: entity

X-LLM-Entity-Typestring(HiddenLayerAnalysisEntityTypeEnum)

The type of entity to redact

Default strict

Enum"strict""all"

Example: strict

X-LLM-Skip-Input-Code-Detectionboolean

Whether to skip input code detection

Default false

X-LLM-Skip-Output-Code-Detectionboolean

Whether to skip output code detection

Default false

X-LLM-Block-Input-Code-Detectionboolean

Whether to block input code detection

Default false

X-LLM-Block-Output-Code-Detectionboolean

Whether to block output code detection

Default false

X-LLM-Skip-Guardrail-Detectionboolean

Whether to skip guardrail detection

Default false

X-LLM-Block-Guardrail-Detectionboolean

Whether to block guardrail detection

Default false

X-LLM-Skip-Input-URL-Detectionboolean

Whether to skip input URL detection

Default false

X-LLM-Skip-Output-URL-Detectionboolean

Whether to skip output URL detection

Default false

Bodyapplication/jsonrequired

One of:

The request to an Azure Chat Completions endpoint

input_dataobject

Example: {"input_string":[{"role":"user","content":"Hello World"}],"params":{"top_p":1,"temperature":1,"max_tokens":256,"do_sample":true,"return_full_text":true}}

curl -i -X POST \
  'https://docs.hiddenlayer.ai/_mock/docs/products/aidr-g/llm_proxy_api/api/v1/proxy/azure/{endpoint_name}/score?region=string' \
  -H 'Content-Type: application/json' \
  -H 'HL-Project-Id: internal-search-chatbot' \
  -H 'X-LLM-Block-Guardrail-Detection: false' \
  -H 'X-LLM-Block-Input-Code-Detection: false' \
  -H 'X-LLM-Block-Input-DOS-Detection: false' \
  -H 'X-LLM-Block-Input-PII: false' \
  -H 'X-LLM-Block-Output-Code-Detection: false' \
  -H 'X-LLM-Block-Output-PII: false' \
  -H 'X-LLM-Block-Prompt-Injection: false' \
  -H 'X-LLM-Block-Unsafe: false' \
  -H 'X-LLM-Block-Unsafe-Input: false' \
  -H 'X-LLM-Block-Unsafe-Output: false' \
  -H 'X-LLM-Entity-Type: strict' \
  -H 'X-LLM-Input-DOS-Detection-Threshold: 4096' \
  -H 'X-LLM-Prompt-Injection-Scan-Type: quick' \
  -H 'X-LLM-Redact-Input-PII: false' \
  -H 'X-LLM-Redact-Output-PII: false' \
  -H 'X-LLM-Redact-Type: entity' \
  -H 'X-LLM-Skip-Guardrail-Detection: false' \
  -H 'X-LLM-Skip-Input-Code-Detection: false' \
  -H 'X-LLM-Skip-Input-DOS-Detection: false' \
  -H 'X-LLM-Skip-Input-PII-Detection: false' \
  -H 'X-LLM-Skip-Input-URL-Detection: false' \
  -H 'X-LLM-Skip-Output-Code-Detection: false' \
  -H 'X-LLM-Skip-Output-PII-Detection: false' \
  -H 'X-LLM-Skip-Output-URL-Detection: false' \
  -H 'X-LLM-Skip-Prompt-Injection-Detection: false' \
  -H 'X-Requester-Id: string' \
  -d '{
    "input_data": {
      "input_string": [
        {
          "role": "user",
          "content": "Hello World"
        }
      ],
      "params": {
        "top_p": 1,
        "temperature": 1,
        "max_tokens": 256,
        "do_sample": true,
        "return_full_text": true
      }
    }
  }'

Responses

Successful chat completions and analysis response

Bodyapplication/json

One of:

responseArray of strings(AzureTextCompletionsResponse)

The response from an Azure Text Completions endpoint

providerstring

The provider of the upstream service

Example: "azure"

modelstring

The name of the model

Example: "azure-endpoint/llama-2-deployment"

verdictboolean

The overall verdict of the analysis

categoriesobject(HiddenLayerAnalysisCategories)

The analysis detection categories

resultsobject(HiddenLayerAnalysisResults)

The analysis results

policyobject(HiddenLayerAnalysisPolicy)

The policy used during analysis

frameworksobject(HiddenLayerAnalysisFrameworks)

The framework labels identified during analysis

elapsed_msnumber

The time in milliseconds it took to process the request

upstream_elapsed_msnumber

The time in milliseconds the upstream LLM took to process the request

Response

application/json

{
  "response": [
    "Hello, how can I help you today?"
  ],
  "provider": "azure",
  "model": "azure-endpoint/llama-2-deployment",
  "verdict": true,
  "categories": {
    "unsafe_input": true,
    "unsafe_output": true,
    "prompt_injection": true,
    "input_dos": true,
    "input_pii": true,
    "output_pii": true,
    "input_code": true,
    "output_code": true,
    "guardrail": true
  },
  "results": {
    "input_block_list_results": { … },
    "prompt_injection_classifier_results": [ … ],
    "input_dos_results": { … },
    "input_pii_results": { … },
    "output_pii_results": { … },
    "input_code_results": { … },
    "output_code_results": { … },
    "guardrail_results": { … },
    "input_urls": { … },
    "output_urls": { … }
  },
  "policy": {
    "block_unsafe": false,
    "block_unsafe_input": false,
    "block_unsafe_output": true,
    "skip_prompt_injection_detection": true,
    "block_prompt_injection": true,
    "prompt_injection_scan_type": "quick",
    "skip_input_pii_detection": true,
    "skip_output_pii_detection": true,
    "block_input_pii": true,
    "block_output_pii": true,
    "redact_input_pii": true,
    "redact_output_pii": true,
    "redact_type": "entity",
    "entity_type": "strict",
    "skip_input_code_detection": true,
    "skip_output_code_detection": true,
    "block_input_code_detection": true,
    "block_output_code_detection": true,
    "skip_guardrail_detection": true,
    "block_guardrail_detection": true,
    "skip_input_url_detection": true,
    "skip_output_url_detection": true,
    "skip_input_dos_detection": true,
    "block_input_dos_detection": true,
    "input_dos_detection_threshold": 4096
  },
  "frameworks": {
    "mitre": [ … ],
    "owasp": [ … ]
  },
  "elapsed_ms": 0,
  "upstream_elapsed_ms": 0
}

Proxy Azure endpoint and deployment using Azure Entra

Request

Generate text or chat completions for a given Azure endpoint and return response and analysis

Path

endpoint_namestringrequired

The name of the Azure endpoint to use

deployment_namestringrequired

The name of the Azure endpoint deployment to use

Query

subscription_idstringrequired

The subscription id of the Azure endpoint

resource_groupstringrequired

The resource group of the Azure endpoint

workspacestringrequired

The workspace of the Azure endpoint

regionstringrequired

The region of the Azure endpoint

Headers

HL-Project-Idstring

The ID or alias for the Project to govern the request processing.

Example: internal-search-chatbot

X-Requester-Idstring

The identifier for the requester to be used if MLDR is enabled

X-LLM-Block-Unsafeboolean

Whether to block unsafe input and output

Default false

X-LLM-Block-Unsafe-Inputboolean

Whether to block unsafe input

Default false

X-LLM-Block-Unsafe-Outputboolean

Whether to block unsafe output

Default false

X-LLM-Skip-Prompt-Injection-Detectionboolean

Whether to skip prompt injection detection

Default false

X-LLM-Block-Prompt-Injectionboolean

Whether to block prompt injection

Default false

X-LLM-Prompt-Injection-Scan-Typestring(HiddenLayerAnalysisPromptInjectionEnum)

The type of prompt injection scan to use

Default quick

Enum"quick""full"

Example: quick

X-LLM-Skip-Input-DOS-Detectionboolean

Whether to skip input denial of service detection

Default false

X-LLM-Block-Input-DOS-Detectionboolean

Whether to block input denial of service detection

Default false

X-LLM-Input-DOS-Detection-Thresholdstring

The threshold for input denial of service detection

Default 4096

X-LLM-Skip-Input-PII-Detectionboolean

Whether to skip input personally identifiable information detection

Default false

X-LLM-Skip-Output-PII-Detectionboolean

Whether to skip output personally identifiable information detection

Default false

X-LLM-Block-Input-PIIboolean

Whether to block input personally identifiable information detection

Default false

X-LLM-Block-Output-PIIboolean

Whether to block output personally identifiable information detection

Default false

X-LLM-Redact-Input-PIIboolean

Whether to redact input personally identifiable information

Default false

X-LLM-Redact-Output-PIIboolean

Whether to redact output personally identifiable information

Default false

X-LLM-Redact-Typestring(HiddenLayerAnalysisRedactTypeEnum)

The type of redaction to use

Default entity

Enum"entity""strict"

Example: entity

X-LLM-Entity-Typestring(HiddenLayerAnalysisEntityTypeEnum)

The type of entity to redact

Default strict

Enum"strict""all"

Example: strict

X-LLM-Skip-Input-Code-Detectionboolean

Whether to skip input code detection

Default false

X-LLM-Skip-Output-Code-Detectionboolean

Whether to skip output code detection

Default false

X-LLM-Block-Input-Code-Detectionboolean

Whether to block input code detection

Default false

X-LLM-Block-Output-Code-Detectionboolean

Whether to block output code detection

Default false

X-LLM-Skip-Guardrail-Detectionboolean

Whether to skip guardrail detection

Default false

X-LLM-Block-Guardrail-Detectionboolean

Whether to block guardrail detection

Default false

X-LLM-Skip-Input-URL-Detectionboolean

Whether to skip input URL detection

Default false

X-LLM-Skip-Output-URL-Detectionboolean

Whether to skip output URL detection

Default false

Bodyapplication/jsonrequired

One of:

The request to an Azure Chat Completions endpoint

input_dataobject

Example: {"input_string":[{"role":"user","content":"Hello World"}],"params":{"top_p":1,"temperature":1,"max_tokens":256,"do_sample":true,"return_full_text":true}}

curl -i -X POST \
  'https://docs.hiddenlayer.ai/_mock/docs/products/aidr-g/llm_proxy_api/api/v1/proxy/azure/{endpoint_name}/{deployment_name}?subscription_id=string&resource_group=string&workspace=string&region=string' \
  -H 'Content-Type: application/json' \
  -H 'HL-Project-Id: internal-search-chatbot' \
  -H 'X-LLM-Block-Guardrail-Detection: false' \
  -H 'X-LLM-Block-Input-Code-Detection: false' \
  -H 'X-LLM-Block-Input-DOS-Detection: false' \
  -H 'X-LLM-Block-Input-PII: false' \
  -H 'X-LLM-Block-Output-Code-Detection: false' \
  -H 'X-LLM-Block-Output-PII: false' \
  -H 'X-LLM-Block-Prompt-Injection: false' \
  -H 'X-LLM-Block-Unsafe: false' \
  -H 'X-LLM-Block-Unsafe-Input: false' \
  -H 'X-LLM-Block-Unsafe-Output: false' \
  -H 'X-LLM-Entity-Type: strict' \
  -H 'X-LLM-Input-DOS-Detection-Threshold: 4096' \
  -H 'X-LLM-Prompt-Injection-Scan-Type: quick' \
  -H 'X-LLM-Redact-Input-PII: false' \
  -H 'X-LLM-Redact-Output-PII: false' \
  -H 'X-LLM-Redact-Type: entity' \
  -H 'X-LLM-Skip-Guardrail-Detection: false' \
  -H 'X-LLM-Skip-Input-Code-Detection: false' \
  -H 'X-LLM-Skip-Input-DOS-Detection: false' \
  -H 'X-LLM-Skip-Input-PII-Detection: false' \
  -H 'X-LLM-Skip-Input-URL-Detection: false' \
  -H 'X-LLM-Skip-Output-Code-Detection: false' \
  -H 'X-LLM-Skip-Output-PII-Detection: false' \
  -H 'X-LLM-Skip-Output-URL-Detection: false' \
  -H 'X-LLM-Skip-Prompt-Injection-Detection: false' \
  -H 'X-Requester-Id: string' \
  -d '{
    "input_data": {
      "input_string": [
        {
          "role": "user",
          "content": "Hello World"
        }
      ],
      "params": {
        "top_p": 1,
        "temperature": 1,
        "max_tokens": 256,
        "do_sample": true,
        "return_full_text": true
      }
    }
  }'

Responses

Successful chat completions and analysis response

Bodyapplication/json

One of:

responseArray of strings(AzureTextCompletionsResponse)

The response from an Azure Text Completions endpoint

providerstring

The provider of the upstream service

Example: "azure"

modelstring

The name of the model

Example: "azure-endpoint/llama-2-deployment"

verdictboolean

The overall verdict of the analysis

categoriesobject(HiddenLayerAnalysisCategories)

The analysis detection categories

resultsobject(HiddenLayerAnalysisResults)

The analysis results

policyobject(HiddenLayerAnalysisPolicy)

The policy used during analysis

frameworksobject(HiddenLayerAnalysisFrameworks)

The framework labels identified during analysis

elapsed_msnumber

The time in milliseconds it took to process the request

upstream_elapsed_msnumber

The time in milliseconds the upstream LLM took to process the request

Response

application/json

{
  "response": [
    "Hello, how can I help you today?"
  ],
  "provider": "azure",
  "model": "azure-endpoint/llama-2-deployment",
  "verdict": true,
  "categories": {
    "unsafe_input": true,
    "unsafe_output": true,
    "prompt_injection": true,
    "input_dos": true,
    "input_pii": true,
    "output_pii": true,
    "input_code": true,
    "output_code": true,
    "guardrail": true
  },
  "results": {
    "input_block_list_results": { … },
    "prompt_injection_classifier_results": [ … ],
    "input_dos_results": { … },
    "input_pii_results": { … },
    "output_pii_results": { … },
    "input_code_results": { … },
    "output_code_results": { … },
    "guardrail_results": { … },
    "input_urls": { … },
    "output_urls": { … }
  },
  "policy": {
    "block_unsafe": false,
    "block_unsafe_input": false,
    "block_unsafe_output": true,
    "skip_prompt_injection_detection": true,
    "block_prompt_injection": true,
    "prompt_injection_scan_type": "quick",
    "skip_input_pii_detection": true,
    "skip_output_pii_detection": true,
    "block_input_pii": true,
    "block_output_pii": true,
    "redact_input_pii": true,
    "redact_output_pii": true,
    "redact_type": "entity",
    "entity_type": "strict",
    "skip_input_code_detection": true,
    "skip_output_code_detection": true,
    "block_input_code_detection": true,
    "block_output_code_detection": true,
    "skip_guardrail_detection": true,
    "block_guardrail_detection": true,
    "skip_input_url_detection": true,
    "skip_output_url_detection": true,
    "skip_input_dos_detection": true,
    "block_input_dos_detection": true,
    "input_dos_detection_threshold": 4096
  },
  "frameworks": {
    "mitre": [ … ],
    "owasp": [ … ]
  },
  "elapsed_ms": 0,
  "upstream_elapsed_ms": 0
}

HuggingFace TGI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Operations

LLM Proxy (1)
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Health and Metrics
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

OpenAI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Azure
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Proxy Azure endpoint

Request

Responses

Was this section helpful?

Proxy Azure endpoint and deployment using Azure Entra

Request

Responses

Was this section helpful?

HuggingFace TGI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

LLM Proxy (1)CopyCopy for LLMCopy page as Markdown for LLMsView as MarkdownOpen this page as MarkdownOpen in ChatGPTGet insights from ChatGPTOpen in ClaudeGet insights from ClaudeConnect to CursorInstall MCP server on CursorConnect to VS CodeInstall MCP server on VS Code

Health and MetricsCopyCopy for LLMCopy page as Markdown for LLMsView as MarkdownOpen this page as MarkdownOpen in ChatGPTGet insights from ChatGPTOpen in ClaudeGet insights from ClaudeConnect to CursorInstall MCP server on CursorConnect to VS CodeInstall MCP server on VS Code

OpenAICopyCopy for LLMCopy page as Markdown for LLMsView as MarkdownOpen this page as MarkdownOpen in ChatGPTGet insights from ChatGPTOpen in ClaudeGet insights from ClaudeConnect to CursorInstall MCP server on CursorConnect to VS CodeInstall MCP server on VS Code

AzureCopyCopy for LLMCopy page as Markdown for LLMsView as MarkdownOpen this page as MarkdownOpen in ChatGPTGet insights from ChatGPTOpen in ClaudeGet insights from ClaudeConnect to CursorInstall MCP server on CursorConnect to VS CodeInstall MCP server on VS Code

Proxy Azure endpoint

RequestExpand all

Responses

Was this section helpful?

Proxy Azure endpoint and deployment using Azure Entra

RequestExpand all

Responses

Was this section helpful?

HuggingFace TGICopyCopy for LLMCopy page as Markdown for LLMsView as MarkdownOpen this page as MarkdownOpen in ChatGPTGet insights from ChatGPTOpen in ClaudeGet insights from ClaudeConnect to CursorInstall MCP server on CursorConnect to VS CodeInstall MCP server on VS Code

LLM Proxy (1)
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Health and Metrics
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

OpenAI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Azure
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code

Request

Request

HuggingFace TGI
Copy for LLM
Copy page as Markdown for LLMs
View as Markdown
Open this page as Markdown
Open in ChatGPT
Get insights from ChatGPT
Open in Claude
Get insights from Claude
Connect to Cursor
Install MCP server on Cursor
Connect to VS Code
Install MCP server on VS Code