Debugging AI Refusal to Answer Questions

When an AI refuses to answer a question, especially when instructed to "Refuse to answer beyond the given knowledge base," it’s often because there is no relevant information available or the question is out of context. However, this refusal should only occur under specific circumstances. Below is a guide to help you troubleshoot and resolve this issue.

1. Verify Data Sources

Check the Response Source

Start by examining the sources of the AI-generated response. If you find data related to your query within your data sources and the information has a decent confidence score, but the AI still refuses to answer, the issue might lie in the prompts you've set.

Adjusting Prompts

If the data is present but not being used correctly, inspect your "Base Prompt" or "Restriction Prompt." These prompts might need tweaking to ensure the AI responds appropriately based on the available data.

2. Missing Information in Data Sources

No Relevant Data Found

If you don’t find any trace of the query’s information in the data sources, it indicates that the AI couldn’t locate the necessary data. This could be due to incomplete training.

Verify Training Data

Check whether you’ve trained the bot on the relevant information. If the data was included in the training but is still not being accessed, the training data might not be properly formatted or well-structured. It’s important to ensure the training data is correctly structured to optimize the AI’s performance.

3. Using the Debug Lab Feature

Search and Investigate

For further investigation, utilize the Debug Lab feature. This tool allows you to search through all your training material and lists all related data sources for any given query.

Filtering Data Points

You can filter data points to specific documents using the Source button in your trained data listing. This functionality helps you retrieve all data sources linked to a particular document or link, enabling you to query and view data points specific to that document.

4. Editing and Adjusting Data Sources

Modifying Data Sources

You have the option to edit these data sources, make adjustments, or remove certain data points as needed. However, keep in mind that any edits made directly to the data sources may be overridden when the related document is re-trained.

Adjusting Training Material

To avoid losing your changes, it's important to adjust the training material itself rather than just the data sources. This ensures that your AI bot consistently has access to the correct information during re-training.

Check out our Training Guide for more tips and best practices.