AIDR GenAI can be configured to route traffic to a Databricks OpenAI instance.
- Replace
<name>with the Databricks OpenAI instance name. Example:databricks-prod. - Replace
<route_prefix>with the route prefix for the model used. Example:/serving-endpoint/gpt-model. - Replace
<base_url>with the URL for the instance. Example:https://upstream.provider.com. - Set the enrich response to true to return the upstream LLM response enriched with HiddenLayer’s detections and analysis. Set to false to disable the HiddenLayer’s detections and analysis.
- Replace
<api_key>with a Databricks API key.
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS="<name>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ROUTE_PREFIX="<route_prefix>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_BASE_URL="<base_url>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ENRICH_RESPONSE="<true|false>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_API_KEY="<api_key>"| Environment Key | Description | Default | Required |
|---|---|---|---|
| HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS | The name of the Databricks OpenAI instance. | False | |
| HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ROUTE_PREFIX | The route prefix for the model used. | False | |
| HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_BASE_URL | The base URL for the instance. | False | |
| HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ENRICH_RESPONSE | Set AIDR for GenAI as a forward or reverse proxy. | False | |
| HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_API_KEY | Databricks API key. | False |