Skip to main content
Manage reusable prompt sets for testing.

​
Create Prompt Set

  1. In the HiddenLayer Console, select Attack Simulation > Prompt Sets.
  2. Click Create Prompt Set +. The Create Prompt Set slide-out displays.
  3. Enter a name for the prompt set.
  4. Optionally, enter a description.
  5. Upload a CSV file containing your prompt set. You can either drag-and-drop the file into the Upload Prompts field or click inside the field to open the file browser.
  6. Click Create Prompt Set.

​
CSV File Example

CSV format requirements:
  • Maximum file size: 10MB
  • Maximum 1,000 prompts
  • Single column: user prompts only
  • Two columns: system prompts (optional), user prompts
  • First row may contain headers: “system prompt, user prompt” or “user prompt”
  • User prompts: max 20KB each
  • System prompts: max 10KB each

​
Example Formats

System Prompt and User Prompt with header
System Prompt and User Prompt without header
User Prompt with header
User Prompt without header

​
Best Practices

  • Compare Versions: Run evaluations on both original and enhanced prompts to measure improvement.
  • Review Failed Attacks: Understanding why attacks failed is as important as knowing which succeeded.
  • Use Appropriate Models: Match the target model to what you’re actually using in production.

​
Run Evaluation

You can run an evaluation using any available prompt set.
  1. In the Console, select Attack Simulation > Prompt Sets.
  2. For the prompt set you want to use, click Run Evaluation. The Create Red Team Evaluation slide-out displays.
  3. Enter a name for the evaluation.
  4. Enter the target system prompt.
  5. Select a target model.
    • Select a model that is similar to what you have in your environment to simulate the attacks against your system prompt.
      Beta ModelsModels marked with a beta designation may be subject to lower usage quotas, limited availability, or ongoing development changes. As a result, these models may exhibit unexpected results, reduced performance, or intermittent failures during testing. Users should account for these limitations when selecting beta models.
  6. Optional: Click Advanced Options to expand the section.
    • Select a project to apply runtime policies to interaction tagging.
      • If no project is selected, the default project is used.
    • Select an execution strategy.
      • Single: Runs each technique once per objective.
      • Random: Runs all techniques plus N additional random techniques.
        • Select the number of additional random techniques.
      • Static prompt set: Uses a predefined set of static prompts for evaluation.
        • Select the prompt set from the drop-down list.
    • Set the maximum number of conversation turns allowed per technique when attempting to achieve an objective. The minimum is one and the maximum is five.
      • The attack simulator will do multi-turn, trying to attack the target for N turns, and then go to the next session.
      • Note: If you selected static_prompt_set, then Attacker Max Turns to Complete Objective is not available.
    • Set the number of independent sessions to run for each technique. The minimum is one and the maximum is five.
      • This is the number of times you want to run the same technique or static prompt.
  7. Click Start Evaluation.
  8. When the evaluation completes, click the green arrow to view the results. See Red Team Evaluation Summary for more information.
Terminate Running EvaulationYou can terminate a running evaluation from the Active Evaluations tab on the Red Teaming page.
  • In the Status column, click the x icon for the running evaluation that you want to terminate.
  • A confirmation message displays. Click Terminate.
  • If the evaluation was terminated successfully, Terminated displays for the status.

​
Pre-Configured Red Team Prompt Sets

Select a prompt set to run red team evaluations using pre-configured adversarial prompts designed to test your AI system’s defenses. To run an evaluation, see Run Evaluation.