# Proxy HuggingFace TGI Endpoint

Proxy chat completions for a given HuggingFace TGI endpoint and return response and analysis

Endpoint: POST /api/v1/proxy/tgi/{name}
Version: 1
Security: 

## Header parameters:

  - `HL-Project-Id` (string)
    The ID or alias for the Project to govern the request processing.
    Example: "internal-search-chatbot"

  - `X-Requester-Id` (string)
    The identifier for the requester to be used if MLDR is enabled

  - `X-LLM-Block-Unsafe` (boolean)
    Whether to block unsafe input and output

  - `X-LLM-Block-Unsafe-Input` (boolean)
    Whether to block unsafe input

  - `X-LLM-Block-Unsafe-Output` (boolean)
    Whether to block unsafe output

  - `X-LLM-Skip-Prompt-Injection-Detection` (boolean)
    Whether to skip prompt injection detection

  - `X-LLM-Block-Prompt-Injection` (boolean)
    Whether to block prompt injection

  - `X-LLM-Prompt-Injection-Scan-Type` (string)
    The type of prompt injection scan to use
    Enum: "quick", "full"

  - `X-LLM-Skip-Input-DOS-Detection` (boolean)
    Whether to skip input denial of service detection

  - `X-LLM-Block-Input-DOS-Detection` (boolean)
    Whether to block input denial of service detection

  - `X-LLM-Input-DOS-Detection-Threshold` (string)
    The threshold for input denial of service detection

  - `X-LLM-Skip-Input-PII-Detection` (boolean)
    Whether to skip input personally identifiable information detection

  - `X-LLM-Skip-Output-PII-Detection` (boolean)
    Whether to skip output personally identifiable information detection

  - `X-LLM-Block-Input-PII` (boolean)
    Whether to block input personally identifiable information detection

  - `X-LLM-Block-Output-PII` (boolean)
    Whether to block output personally identifiable information detection

  - `X-LLM-Redact-Input-PII` (boolean)
    Whether to redact input personally identifiable information

  - `X-LLM-Redact-Output-PII` (boolean)
    Whether to redact output personally identifiable information

  - `X-LLM-Redact-Type` (string)
    The type of redaction to use
    Enum: "entity", "strict"

  - `X-LLM-Entity-Type` (string)
    The type of entity to redact
    Enum: "strict", "all"

  - `X-LLM-Skip-Input-Code-Detection` (boolean)
    Whether to skip input code detection

  - `X-LLM-Skip-Output-Code-Detection` (boolean)
    Whether to skip output code detection

  - `X-LLM-Block-Input-Code-Detection` (boolean)
    Whether to block input code detection

  - `X-LLM-Block-Output-Code-Detection` (boolean)
    Whether to block output code detection

  - `X-LLM-Skip-Guardrail-Detection` (boolean)
    Whether to skip guardrail detection

  - `X-LLM-Block-Guardrail-Detection` (boolean)
    Whether to block guardrail detection

  - `X-LLM-Skip-Input-URL-Detection` (boolean)
    Whether to skip input URL detection

  - `X-LLM-Skip-Output-URL-Detection` (boolean)
    Whether to skip output URL detection

## Path parameters:

  - `name` (string, required)
    The name of the TGI endpoint defined in service settings

## Request fields (application/json):

  - `model` (string)
    The model to use for completions

  - `messages` (array)

  - `messages.role` (string)
    The role of the message

  - `messages.content` (string)
    The content of the message

  - `temperature` (number)
    The temperature to use for completions

  - `max_tokens` (number)
    The maximum number of tokens to generate

  - `top_p` (number)
    The top p value to use for completions

  - `frequency_penalty` (number)
    The frequency penalty to use for completions

  - `presence_penalty` (number)
    The presence penalty to use for completions

## Response 200 fields (application/json):

  - `body` (HiddenLayerProxyHuggingFaceTGIResponse (object))
    - `response` (object)
      The response from an OpenAI Chat Completions endpoint
    - `response.choices` (array)
    - `response.choices.finish_reason` (string)
      The reason the completion finished
    - `response.choices.index` (number)
      The index of the completion
    - `response.choices.message` (array)
    - `response.choices.message.role` (string)
      The role of the message
    - `response.choices.message.content` (string)
      The content of the message
    - `response.choices.logprobs` (null,string)
      The logprobs of the completion
    - `response.created` (number)
      The timestamp of the completion
    - `response.id` (null,string)
      The id of the completion
    - `response.model` (string)
      The model used for the completion
    - `response.object` (string)
      The object of the completion
    - `response.usage` (object)
    - `response.usage.completion_tokens` (number)
      The number of completion tokens
    - `response.usage.prompt_tokens` (number)
      The number of prompt tokens
    - `response.usage.total_tokens` (number)
      The total number of tokens
    - `provider` (string)
      The provider of the upstream service
      Example: "huggingface-tgi"
    - `model` (string)
      The name of the model
      Example: "tgi-model"
    - `verdict` (boolean)
      The overall verdict of the analysis
    - `categories` (object)
      The analysis detection categories
    - `categories.unsafe_input` (boolean)
      The input is unsafe
    - `categories.unsafe_output` (boolean)
      The output is unsafe
    - `categories.prompt_injection` (boolean)
      The input contains prompt injection
    - `categories.input_dos` (boolean)
      The input contains a denial of service attack
    - `categories.input_pii` (boolean)
      The input contains personally identifiable information
    - `categories.output_pii` (boolean)
      The output contains personally identifiable information
    - `categories.input_code` (boolean)
      The input contains code
    - `categories.output_code` (boolean)
      The output contains code
    - `categories.guardrail` (boolean)
      The input activated the upstream guardrails
    - `results` (object)
      The analysis results
    - `results.input_block_list_results` (object)
      The input block list results
      Example: {"verdict":true,"matches":["badToken"],"elapsed_ms":10}
    - `results.input_block_list_results.verdict` (boolean)
      The verdict of the input block list analysis
      Example: true
    - `results.input_block_list_results.matches` (array)
      Example: ["badToken"]
    - `results.input_block_list_results.elapsed_ms` (number)
      The time in milliseconds it took to process the input block list
      Example: 10
    - `results.prompt_injection_classifier_results` (array)
      Example: [{"version":1,"verdict":true,"probabilities":[1],"elapsed_ms":100}]
    - `results.prompt_injection_classifier_results.version` (number)
      The version of the prompt injection classifier
    - `results.prompt_injection_classifier_results.verdict` (boolean)
      The verdict of the prompt injection classifier
    - `results.prompt_injection_classifier_results.probabilities` (array)
    - `results.prompt_injection_classifier_results.elapsed_ms` (number)
      The time in milliseconds it took to process the prompt injection classifier
    - `results.input_dos_results` (object)
      The input denial of service results
    - `results.input_dos_results.verdict` (boolean)
      The verdict of the input denial of service analysis
    - `results.input_dos_results.elapsed_ms` (number)
      The time in milliseconds it took to process the input denial of service
    - `results.input_pii_results` (object)
      The input personally identifiable information results
      Example: {"verdict":true,"entities":["PHONE_NUMBER"],"elapsed_ms":20}
    - `results.input_pii_results.verdict` (boolean)
      The verdict of the input personally identifiable information analysis
      Example: true
    - `results.input_pii_results.entities` (array)
      Example: ["PHONE_NUMBER"]
    - `results.input_pii_results.elapsed_ms` (number)
      The time in milliseconds it took to process the input personally identifiable information
      Example: 20
    - `results.output_pii_results` (object)
      The output personally identifiable information results
      Example: {"verdict":true,"entities":["LOCATION"],"elapsed_ms":20}
    - `results.output_pii_results.verdict` (boolean)
      The verdict of the output personally identifiable information analysis
      Example: true
    - `results.output_pii_results.entities` (array)
      Example: ["LOCATION"]
    - `results.output_pii_results.elapsed_ms` (number)
      The time in milliseconds it took to process the output personally identifiable information
      Example: 20
    - `results.input_code_results` (object)
      The input code results
    - `results.input_code_results.verdict` (boolean)
      The verdict of the input code analysis
    - `results.input_code_results.elapsed_ms` (number)
      The time in milliseconds it took to process the input code
    - `results.output_code_results` (object)
      The output code results
    - `results.output_code_results.verdict` (boolean)
      The verdict of the output code analysis
    - `results.output_code_results.elapsed_ms` (number)
      The time in milliseconds it took to process the output code
    - `results.guardrail_results` (object)
      The guardrail results
    - `results.guardrail_results.verdict` (boolean)
      The verdict of the guardrail analysis
    - `results.guardrail_results.elapsed_ms` (number)
      The time in milliseconds it took to process the guardrail
    - `results.input_urls` (object)
      The input URL results
    - `results.input_urls.urls` (array)
    - `results.input_urls.elapsed_ms` (number)
      The time in milliseconds it took to process the guardrail
    - `results.output_urls` (object)
      The output URL results
    - `results.output_urls.urls` (array)
    - `results.output_urls.elapsed_ms` (number)
      The time in milliseconds it took to process the guardrail
    - `policy` (object)
      The policy used during analysis
    - `policy.block_unsafe` (boolean)
      Block unsafe input and output
    - `policy.block_unsafe_input` (boolean)
      Block unsafe input
    - `policy.block_unsafe_output` (boolean)
      Block unsafe output
    - `policy.skip_prompt_injection_detection` (boolean)
      Skip prompt injection detection
    - `policy.block_prompt_injection` (boolean)
      Block prompt injection
    - `policy.prompt_injection_scan_type` (string)
      The type of prompt injection scan to use
      Enum: same as `X-LLM-Prompt-Injection-Scan-Type` (2 values)
    - `policy.skip_input_pii_detection` (boolean)
      Skip input personally identifiable information detection
    - `policy.skip_output_pii_detection` (boolean)
      Skip output personally identifiable information detection
    - `policy.block_input_pii` (boolean)
      Block input personally identifiable information
    - `policy.block_output_pii` (boolean)
      Block output personally identifiable information
    - `policy.redact_input_pii` (boolean)
      Redact input personally identifiable information
    - `policy.redact_output_pii` (boolean)
      Redact output personally identifiable information
    - `policy.redact_type` (string)
      The type of redaction to use
      Enum: same as `X-LLM-Redact-Type` (2 values)
    - `policy.entity_type` (string)
      The type of entity to redact
      Enum: same as `X-LLM-Entity-Type` (2 values)
    - `policy.skip_input_code_detection` (boolean)
      Skip input code detection
    - `policy.skip_output_code_detection` (boolean)
      Skip output code detection
    - `policy.block_input_code_detection` (boolean)
      Block input code detection
    - `policy.block_output_code_detection` (boolean)
      Block output code detection
    - `policy.skip_guardrail_detection` (boolean)
      Skip guardrail detection
    - `policy.block_guardrail_detection` (boolean)
      Block guardrail detection
    - `policy.skip_input_url_detection` (boolean)
      Skip input URL detection
    - `policy.skip_output_url_detection` (boolean)
      Skip output URL detection
    - `policy.skip_input_dos_detection` (boolean)
      Skip input denial of service detection
    - `policy.block_input_dos_detection` (boolean)
      Block input denial of service detection
    - `policy.input_dos_detection_threshold` (number)
      The threshold for input denial of service detection
      Example: 4096
    - `frameworks` (object)
      The framework labels identified during analysis
    - `frameworks.mitre` (array)
      Example: [{"name":"LLM Prompt Injection","label":"AML.T0051"}]
    - `frameworks.mitre.name` (string)
      The name of the MITRE Atlas framework label
    - `frameworks.mitre.label` (string)
      The label of the MITRE Atlas framework label
    - `frameworks.owasp` (array)
      Example: [{"name":"Prompt Injection","label":"LLM01"}]
    - `frameworks.owasp.name` (string)
      The name of the OWASP framework label
    - `frameworks.owasp.label` (string)
      The label of the OWASP framework label
    - `elapsed_ms` (number)
      The time in milliseconds it took to process the request
    - `upstream_elapsed_ms` (number)
      The time in milliseconds the upstream LLM took to process the request

## Response 502 fields (application/json):

  - `detail` (string)