Retrieval-Augmented Generation (RAG) combines the strengths of traditional information retrieval systems (such as databases) with the capabilities of generative large language models (LLMs). By integrating external knowledge with its skills, the AI can produce answers that are more accurate, up-to-date, and relevant to specific needs. If you want to understand the basics of RAG take a look at this article.
Why is it called RAG?
Patrick Lewis, the primary author of the paper that first presented RAG in 2020, named the acronym that currently characterizes an expanding array of techniques utilized in countless papers and multiple commercial services. He believes these represent the forthcoming evolution of generative AI.
Patrick Lewis leads a team at AI startup Cohere. He talked about how they came up with the name in an interview in Singapore where he shared his ideas with a conference of database developers in the region.
“We always planned to have a nicer sounding name, but when it came time to write the paper, no one had a better idea.” Lewis said.
For more on the beginnings of RAG, watch this interview of Lewis:
https://www.youtube.com/watch?v=Dm5sfALoL1Y
The History of Retrieval Augmented Generation (RAG)
Retrieval Augmented Generation (RAG) is a technique that combines the strengths of information retrieval and natural language generation to produce highly relevant and accurate responses to user queries. The roots of RAG trace back to the early 1970s, a period marked by the pioneering efforts in developing question-answering systems.
The Early Beginnings: 1970s
In the 1970s, researchers began exploring information retrieval and natural language processing (NLP) to develop systems that could answer questions posed in natural language. These early question-answering systems were rudimentary and focused on narrow domains, such as baseball statistics. Despite their limited scope, these systems laid the foundational concepts for future advancements in text mining and information retrieval. We highly recommend the movie Moneyball to see a clear example of using statistics for prediction in sports.
Steady Progress: 1980s and 1990s
A notable milestone during this era was the launch of Ask Jeeves in the mid-1990s. This service, now known as Ask.com, popularized question-answering by allowing users to ask questions in natural language. After this, many other platforms created alternatives like search engines (google, yahoo) and question-based sites like Reddit and Quora.
Breakthroughs in the New Millennium: 2000s and 2010s
The early 2000s and 2010s witnessed groundbreaking advancements in question-answering systems. In 2011, IBM’s Watson earned its fame for defeating two human champions on the TV show Jeopardy!. This was a significant accomplishment. Watson’s success demonstrated the potential of combining information retrieval with advanced NLP and machine learning techniques.
So Who Invented Retrieval Augmented Generation?
Since its publication, hundreds of papers have cited the Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks paper, building upon and expanding its concepts to make it a significant contribution to ongoing research in this field.
In 2020, the paper was published while Lewis was pursuing his doctorate in NLP at University College London and working for Meta at a new AI lab in London. The team aimed to enhance the knowledge capacity of large language models (LLMs) and developed a benchmark to measure their progress.
Drawing inspiration from previous methods and a paper by Google researchers, the team envisioned a trained system with an embedded retrieval index that could learn and generate any desired text output, according to Lewis.
What is Retrieval Augmented Generation?
Keep reading here: https://www.valere.io/blog-post/retrieval-augmented-generation-rag-ultimate-guide/120
