Skip to main content
System Prompt Evaluation helps you build comprehensive, secure system prompts through an interactive AI-guided analysis process.

​
Best Practices

  • Be Specific in Answers: Provide detailed responses to AI questions for better results.
  • Start Simple: Begin with three questions, increase if needed.
  • Iterate: Run multiple evaluations as requirements evolve.
  • Review Carefully: All suggestions do not have to be accepted.
  • Use Presets to Learn: Try preset scenarios to understand common vulnerabilities.

​
Run New Evaluation

  1. In the HiddenLayer Console, select Attack Simulation > System Prompt.
  2. Click Run New Evaluation. The Create System Prompt Evaluation slide-out displays.
  3. Enter a name for the evaluation.
  4. Enter the target system prompt.
  5. For Analysis Configuration, select the number of questions you will answer for this evaluation.
    • The purpose of the evaluation questions is to try and tailor the new system prompt to the organization’s concerns and context.
    • The minimum is one and the maximum is five.
    • Selecting fewer questions means less guidance to customization on your system prompt. Selecting more questions guides the simulation on more customization toward security best practices.
  6. Click Next. The analysis is started. Questions are generated; this may take a moment.
    • You can leave the slide-out and return later to answer the questions.
    • Note: You must return to the evaluation within 60 minutes, otherwise the evaluation process will timeout and you must start over.
  7. Answer the questions based on your organization’s guidelines and goals.
  8. Click Submit. The analysis may take a moment.
    • An estimated processing time displays.
    • More questions/answers in the evaluation will take more time to process.
    • The elapsed time includes time spent away from the evaluation.
  9. When the evaluation completes, click the green arrow to view the results. See System Prompt Evaluation Summary for more information.

​
Active Evaluations

Active evaluations show running jobs and recently completed evaluations (last 90 days). This provides an operational view of what is running and what just finished. For a historical view that includes results older than 90 days, see Evaluation Results

​
Filter Results

  1. Click Filter. The Filters slide-out displays.
  2. Select the statuses you want to view.
  3. Click Show Results.

​
Active Evaluation Table Descriptions

​
Evaluation Results

Evaluation results show all completed evaluation results (no date constraints). This provides a historical view of completed evaluations. To see evaluations that are currently running, see Active Evaluations.

​
Evaluation Results Table Descriptions