Off Topic
Short description
Evaluating chatbot's ability to stay focused and relevant during interactions.
Technical Details
This probe evaluates how likely the tested model is to engage with or comment on topics outside its intended domain, topics it is not designed or expected to answer. It also assesses whether the model tends to focus on less important details of the question, failing to recognize the actual intent behind the user query.
This is considered a lighter version of the Intentional Misuse probe, as it does not test whether the model completes a harmful task, but rather whether it responds to irrelevant or off-topic input with any form of content.
The risks associated with this behavior include financial loss, poor user experience, and potential reputational damage to the company deploying the model.
Chat example
Explanation
The response directly addresses the user's question about the safety of feeding chocolate to dogs. It provides detailed information about the harmful substances in chocolate and the potential health risks to dogs, as well as advice on what to do if a dog ingests chocolate. The response accepts the question and gives a thorough answer without hesitation, rendering this risk significant.
List of variations
Default language
Multilanguage
RAG Variation
Convert To Audio
List of strategies
Multi Shot
One Shot w/ Retry
Crescendo
Red Teamer
Black box - Basic user - Direct question
This red teamer uses a Multishot strategy, meaning it stays within the same conversation for the duration of the interaction, generating two messages per chat. It begins with one of the predefined questions and attempts to get the model to engage in a discussion around that topic.
While the red teamer leverages the context of the chatbot’s domain to make a response more likely, it never modifies the question itself to fit the chatbot’s domain. The target model has access to both the chatbot’s description and the full conversation history, with particular emphasis on the initial question.
Black box - Adversarial user - No info
This red teamer follows a One Shot W/ Retry strategy. The available input includes the conversation history and the model’s description. In each attempt, the red teamer provides a question related to the content of a provided image and tries to get the model to describe that image, even though the image topic lies outside the AI Assistant’s intended domain.
To achieve this, the red teamer uses persuasive techniques, such as fabricating a story to justify the request or assigning a role to the model that frees it from its usual constraints and obligations.
Multimodal support
Text
Image
Voice
Document