Provides summary details, like the number of requests and tokens. Also provides the original system prompt, the recommended compiled system prompt, and the security updates provided by the recommended system prompt.
Run a Red Team Evaluation against the original system prompt or the compiled system prompt.
System Prompt Evaluation data is retained for 90 days.
The initial system prompt, provided by the user, when the evaluation was created.
Click the action button (three vertical dots) to view the Actions menu.
To view to a system prompt evaluation summary, select Security Testing > System Prompt, then click the green arrow for the system prompt that you want to see the evaluation summary for.
To create a red team evaluation using the original system prompt, click Run Red Team Evaluation. The Create Red Team Evaluation slide-out displays.
Enter a name for the evaluation.
Select a target model from the drop-down menu.
- Select a model that is similar to what you have in your environment to simulate the attacks against your system prompt.
- Disclaimer: Models marked with a
betadesignation may be subject to lower usage quotas, limited availability, or ongoing development changes. As a result, these models may exhibit unexpected results, reduced performance, or intermittent failures during testing. Users should account for these limitations when selecting beta models.
Optionally, click Advanced Options.
Select a project to apply runtime rulesets to interaction tagging.
- If no project is selected, the default project is used.
Select an execution strategy.
- Single: Runs each technique once per objective.
- Random: Runs all techniques plus N additional random techniques.
- Static prompt set: Uses a predefined set of static prompts for evaluation.
Set the maximum number of conversation turns allowed per technique when attempting to achieve an objective. The minimum is one and the maximum is five.
Set the number of independent sessions to run for each technique. The minimum is one and the maximum is five.
Click Start Evaluation.
Go to Security Testing > Red Teaming. You should see your red team evaluation in the table. A percentage complete and an estimated time to completion display for the evaluation.
When the evaluation is complete, click the green arrow to view the details.
Expand the Analysis Questions to see the questions asked and the answers provided (by the user) when creating this evaluation.
The compiled system prompt is the enhanced version of the system prompt, with security improvements.
The compiled system prompt is the recommended system prompt to use.
When using a compiled prompt, make sure you are following your organization's policies and procedures for system prompts.
Click the action button (three vertical dots) to view the Actions menu.
Click View Diff to see a prompt comparison between the original and compiled prompts.
- Removed content is highlighted in red.
- Added content is highlighted in green.
- Unchanged content is not highlighted.
To view to a system prompt evaluation summary, select Security Testing > System Prompt, then click the green arrow for the system prompt that you want to see the evaluation summary for.
To create a red team evaluation using the original system prompt, click Run Red Team Evaluation. The Create Red Team Evaluation slide-out displays.
Enter a name for the evaluation.
Select a target model from the drop-down menu.
- Select a model that is similar to what you have in your environment to simulate the attacks against your system prompt.
- Note: A model with the
betatag indicates that it might have a lower quota or is a new model (from the provider) that could cause unexpected results or failures.
Optionally, click Advanced Options.
Select a project to apply runtime rulesets to interaction tagging.
- If no project is selected, the default project is used.
Select an execution strategy.
- Single: Runs each technique once per objective.
- Random: Runs all techniques plus N additional random techniques.
- Static prompt set: Uses a predefined set of static prompts for evaluation.
Set the maximum number of conversation turns allowed per technique when attempting to achieve an objective. The minimum is one and the maximum is five.
Set the number of independent sessions to run for each technique. The minimum is one and the maximum is five.
Click Start Evaluation.
Go to Security Testing > Red Teaming. You should see your red team evaluation in the table. A percentage complete and an estimated time to completion display for the evaluation.
When the evaluation is complete, click the green arrow to view the details.
Expand the Security Updates to see what improvements are achieved with the compiled prompt.
| Usage | Description |
|---|---|
| Total Requests | The total number of requests for this evaluation. |
| Total Tokens | The total number of tokens used for this evaluation, both prompt and completion tokens. |
| Prompt Tokens | The number of prompt tokens used for this evaluation. |
| Completion Tokens | The number of completion tokens used for this evaluation. |