Implement Robust Character Guardrails to Prevent Prompt Injections and Jailbreaks

[cccaballero]

Currently, chat users can easily manipulate the LLM bot's behavior through prompt injections and jailbreak techniques. This allows them to override the bot's intended character, access restricted information, or force the bot to generate harmful content.

Goal:

Implement robust guardrails to prevent users from altering the LLM's character and maintain the intended behavior and safety of the bot.

Possible Solutions:

1. Input Filtering and Sanitization:

   • Implement strict input validation to detect and neutralize potential injection attempts.
   • Use regular expressions or dedicated libraries to identify and filter out common injection patterns.
   • Sanitize user input to remove potentially harmful characters or code snippets.

2. Contextual Awareness and Role Reinforcement:

   • Explicitly define the bot's role and persona within the system prompt.
   • Periodically re-assert the bot's intended role during the conversation to reinforce its character.
   • Implement a system to detect and respond to attempts to deviate from the defined role.

3. Prompt Structure and Control:

   • Use clear and structured prompts that minimize the possibility of user manipulation.
   • Separate user input from system instructions to prevent interference.
   • Utilize techniques like delimiters or special tokens to clearly distinguish between user and system content.

4. User Input Evaluation with Specialized Models:

   • Integrate a model like Llama Guard (or similar) to evaluate user messages for potential prompt injection or harmful content.
   • Use the model's output to determine if the message should be allowed or blocked.

Additional Notes:

• Consider using techniques like "prompt prefixing" or "role-playing templates" (We are already doing this, but it can be improved) to reinforce the bot's character.
• Explore the use of dedicated libraries or services for prompt injection detection and prevention.
• This feature is critical for the safety and reliability of the chat application.