To ensure the safety of your generative inputs and outputs, implement a call to Prompt Analyzer when a prompt is received. If the verdict is false, pass the prompt to your application.
When the application returns its output, call Prompt Analyzer again to check whether the model output is safe to pass to the original requester.
- Prompt Analyzer is designed to take one message per call, not a collection of messages.
- Prompt Analyzer can take just input, just output, or both. To gate content separately, make separate calls.
- As a best practice, avoid including the system prompt in the input prompt. The
promptfield should ideally contain only end-user input. Including the system prompt may cause false positive detections, as the classifiers are optimized for user-generated content.