Google

In RAG systems, when retrieved texts are insufficient to form a complete answer, an alternative strategy is to allow the LLM to generate a response using its own pre-trained knowledge. This approach contrasts with strictly limiting the model to the provided context or having it refuse to answer.

Using Pre-trained Knowledge when Retrieved Context is Insufficient

A financial advisory chatbot is built using a retrieval-augmented architecture. It is designed to provide advice based on a secure, internal database of the firm's current investment strategies. A user asks, "What are some general principles for diversifying a retirement portfolio?" This topic is not explicitly covered in the firm's proprietary strategy documents, so the chatbot currently responds, "I do not have information on that topic." How could the system's response strategy be altered to provide a more helpful, yet responsible, answer in this situation?

RAG System Response Strategy

A software company has a chatbot to help engineers with technical questions, powered by a system that searches internal documentation. An engineer asks for a solution to a brand-new software bug that is not yet in the documentation, so the system finds no relevant information. 

Evaluate the potential risks and benefits of two different response strategies for the chatbot in this scenario:
1. The chatbot states it cannot find the information and refuses to answer.
2. The chatbot uses its general, pre-trained knowledge to propose a potential solution, clearly stating that the information is not from the official documentation.

Based on your evaluation, which strategy is more appropriate for this specific use case and why?

RAG Chatbot Response Strategy for Undocumented Issues

A user asks a chatbot, which is designed to answer questions based on a specific corporate knowledge base, 'What were the key findings from the global tech summit that concluded yesterday?' The knowledge base was last updated a week ago and contains no information about the summit. The chatbot responds with a detailed, accurate summary of the summit's key findings. Which statement best evaluates this interaction?

Learn Before

Related