Skip to main content
Provides summary details, like the number of requests and tokens. Also provides the original system prompt, the recommended compiled system prompt, and the security updates provided by the recommended system prompt. Run a Red Team Evaluation against the original system prompt or the compiled system prompt.
Data RetentionSystem Prompt Evaluation data is retained for 90 days.

​
Original System Prompt

The initial system prompt, provided by the user, when the evaluation was created.

​
Actions

Click the action button (three vertical dots) to view the Actions menu.

​
Create Red Team Evaluation using Original System Prompt

  1. In the HiddenLayer Console, select Attack Simulation > System Prompt.
  2. Click the green arrow for the system prompt that you want to see the evaluation summary for.
  3. To create a Red Team Evaluation using the original system prompt, click Run Red Team Evaluation. The Create Red Team Evaluation slide-out displays.
  4. Enter a name for the evaluation.
  5. Select a target model from the drop-down menu.
    • Select a model that is similar to what you have in your environment to simulate the attacks against your system prompt.
    • Disclaimer: Models marked with a beta designation may be subject to lower usage quotas, limited availability, or ongoing development changes. As a result, these models may exhibit unexpected results, reduced performance, or intermittent failures during testing. Users should account for these limitations when selecting beta models.
  6. Optionally, click Advanced Options.
    • Select a project to apply runtime policies to interaction tagging.
      • If no project is selected, the default project is used.
    • Select an execution strategy.
      • Single: Runs each technique once per objective.
      • Random: Runs all techniques plus N additional random techniques.
      • Static prompt set: Uses a predefined set of static prompts for evaluation.
    • Set the maximum number of conversation turns allowed per technique when attempting to achieve an objective. The minimum is one and the maximum is five.
    • Set the number of independent sessions to run for each technique. The minimum is one and the maximum is five.
  7. Click Start Evaluation.
  8. Go to Attack Simulation > Red Teaming. You should see your red team evaluation in the table. A percentage complete and an estimated time to completion display for the evaluation.
  9. When the evaluation is complete, click the green arrow to view the details.

​
Analysis Questions

Expand the Analysis Questions to see the questions asked and the answers provided (by the user) when creating this evaluation.

​
Compiled System Prompt

The compiled system prompt is the enhanced version of the system prompt, with security improvements. The compiled system prompt is the recommended system prompt to use.
Policies and ProceduresWhen using a compiled prompt, make sure you are following your organization’s policies and procedures for system prompts.

​
Actions

Click the action button (three vertical dots) to view the Actions menu.

​
View Diff

Click View Diff to see a prompt comparison between the original and compiled prompts.
  • Removed content is highlighted in red.
  • Added content is highlighted in green.
  • Unchanged content is not highlighted.

​
Create Red Team Evaluation using Original System Prompt

  1. In the Console, select Attack Simulation > System Prompt.
  2. Click the green arrow for the system prompt that you want to see the evaluation summary for.
  3. To create a red team evaluation using the original system prompt, click Run Red Team Evaluation. The Create Red Team Evaluation slide-out displays.
  4. Enter a name for the evaluation.
  5. Select a target model from the drop-down menu.
    • Select a model that is similar to what you have in your environment to simulate the attacks against your system prompt.
    • Note: A model with the beta tag indicates that it might have a lower quota or is a new model (from the provider) that could cause unexpected results or failures.
  6. Optionally, click Advanced Options.
    • Select a project to apply runtime policies to interaction tagging.
      • If no project is selected, the default project is used.
    • Select an execution strategy.
      • Single: Runs each technique once per objective.
      • Random: Runs all techniques plus N additional random techniques.
      • Static prompt set: Uses a predefined set of static prompts for evaluation.
    • Set the maximum number of conversation turns allowed per technique when attempting to achieve an objective. The minimum is one and the maximum is five.
    • Set the number of independent sessions to run for each technique. The minimum is one and the maximum is five.
  7. Click Start Evaluation.
  8. Go to Attack Simulation > Red Teaming. You should see your red team evaluation in the table. A percentage complete and an estimated time to completion display for the evaluation.
  9. When the evaluation is complete, click the green arrow to view the details.

​
Security Updates

Expand the Security Updates to see what improvements are achieved with the compiled prompt.

​
Usage Descriptions