An Innovative Hybrid Rag Solution for Boosting the Performance of an AI Assistant in Finance

Retrieval Augmented Generation (RAG) is a key technique in various generative AI applications, such as chatbots or AI assistants, for retrieving information from a massive corpus and generating comprehensive answers. Traditional RAG approaches rely heavily on vector similarity search, which often result in incorrect, incomplete, or out-of-date answers. Graph RAG leverages knowledge graphs to retrieve contextually relevant nodes and paths for LLM reasoning, which can improve the accuracy and quality of the answers to a certain degree, but creating and maintaining knowledge graphs can be quite costly and time-consuming, and new errors may be introduced if there’s any inaccurate information in knowledge graphs.

We designed and deployed an innovative end-to-end Hybrid RAG solution to enhance our chatbot’s performance, which integrates key entities from our knowledge graph with vector similarity based RAG, plus other LLM based techniques, including RAG based Name Entity Recognition (NER), Date Standardization, Orchestration/Intent Classification, Key Intent prompting, Solr + Vector similarity search, and multi-step reranking. The new solution boosts the accuracy of the chatbot by almost 100%, from ~40% to ~ 80%. This solution also sets up a fundamental framework for the future agentic infrastructure.