Contextual AI, a Silicon Valley-based startup, has introduced a groundbreaking platform called RAG 2.0, which promises to revolutionize retrieval-augmented generation (RAG) for enterprises. According to the NVIDIA Blog, RAG 2.0 achieves approximately 10x better parameter accuracy and performance compared to competing offerings.

Background and Development

Douwe Kiela, CEO of Contextual AI, has been an influential figure in the field of large language models (LLMs). Inspired by seminal papers from Google and OpenAI, Kiela and his team recognized early on the limitations of LLMs in dealing with real-time data. This understanding led to the development of the first RAG architecture in 2020.

RAG is designed to continuously update foundation models with new, relevant information. This approach addresses the data freshness issues inherent in LLMs, making them more useful for enterprise applications. Kiela’s team realized that without efficient and cost-effective access to real-time data, even the most sophisticated LLMs would fall short in delivering value to enterprises.

RAG 2.0: The Next Evolution

Contextual AI’s latest offering, RAG 2.0, builds upon the original architecture to deliver enhanced performance and accuracy. The platform integrates real-time data retrieval with LLMs, enabling a 70-billion-parameter model to run on infrastructure designed for just 7 billion parameters without compromising accuracy. This optimization opens up new possibilities for edge use cases, where smaller, more efficient computing resources are essential.

“When ChatGPT was released, it exposed the limitations of existing LLMs,” explained Kiela. “We knew that RAG was the solution to many of these problems, and we were confident we could improve upon our initial design.”

Integrated Retrievers and Language Models

One of the key innovations in RAG 2.0 is the close integration of its retriever architecture with the LLM. The retriever processes user queries, identifies relevant data sources, and feeds this information back to the LLM, which then generates a response. This integrated approach ensures higher precision and response quality, reducing the likelihood of “hallucinated” data.

Contextual AI differentiates itself by refining its retrievers through back propagation, aligning both retriever and generator components. This unification allows for synchronized adjustments, leading to significant gains in performance and accuracy.

Tackling Complex Use Cases

RAG 2.0 is designed to be LLM-agnostic, compatible with various open-source models like Mistral and Llama. The platform leverages NVIDIA’s Megatron LM and Tensor Core GPUs to optimize its retrievers. Contextual AI employs a “mixture of retrievers” approach to handle data in various formats, such as text, video, and PDF.

This hybrid method involves deploying different types of RAGs and a neural reranking algorithm to prioritize the most relevant information. This approach ensures that the LLM receives the best possible data to generate accurate responses.

“Our hybrid retrieval strategy maximizes performance by leveraging the strengths of different RAG types,” Kiela said. “This flexibility allows us to tailor solutions to specific use cases and data formats.”

The optimized architecture of RAG 2.0 reduces latency and lowers compute demands, making it suitable for a wide range of industries, from fintech and manufacturing to medical devices and robotics. The platform can be deployed in the cloud, on-premises, or in fully disconnected environments, offering versatility to meet diverse enterprise needs.

“We are focused on solving the most challenging use cases,” Kiela added. “Our aim is to enhance high-value, knowledge-intensive roles, enabling companies to save money and boost productivity.”

Image source: Shutterstock