📝 Retrieval Augmented Generation (RAG)

Description

< What is it? >

Retrieval Augmented Generation (RAG) is a technique that combines the power of large language models (LLMs) with external knowledge sources to improve the quality and relevance of generated content. By retrieving relevant information from a knowledge base or database, RAG enhances the model's ability to generate accurate and contextually appropriate responses.

Langchain helps to implement RAG by providing tools and frameworks for integrating LLMs with external data sources, enabling developers to build applications that leverage both the generative capabilities of LLMs and the structured information from databases.

< Main advantages >

1. Up-to-date knowledge — you don't need to retrain the LLM when documents change.
2. Private knowledge — company documents can be supplied at query time.
3. Reduced hallucination — the model can ground its answer in retrieved documents.
4. Citations/sources — you can tell the user which documents support the answer.

Key points

• Data Preparation
  • Load docs (.txt, .pdf, csv, html, etc.)
    • Split into passages (e.g., 100–500 words)
      • Semantic splitting (e.g., using sentence embeddings) is preferred over naive splitting (e.g., by 100 tokens or 1 paragraph).
        • Semantic Chunking for RAG (by Plaban Nayak)

        • # Instantiate an OpenAI embeddings model
          embedding_model = OpenAIEmbeddings(api_key="<OPENAI_API_TOKEN>", model='text-embedding-3-small')
          
          # Create the semantic text splitter with desired parameters
          semantic_splitter = SemanticChunker(
            embeddings=embedding_model, breakpoint_threshold_type="gradient", breakpoint_threshold_amount=0.8
          )
          
          # Split the document
          chunks = semantic_splitter.split_documents(document)

  • Embed each passage into a vector space (e.g., OpenAI embeddings)
  • Store the embeddings in a vector database (e.g., Pinecone, Weaviate, Milvus, Chroma, FAISS)
• Graph Database
  • Neo4j is a graph database that can be used to store and query relationships between entities in the knowledge base. It allows for efficient retrieval of relevant information based on the relationships between entities.
  • LLMGraphTransformer
  • LangChain Cypher search
• RAG evaluation
  • Evaluate a RAG application (by langsmith)
  • Application-specific evaluation approaches (by langsmith)
  • Evaluating RAG pipelines with Ragas + LangSmith (by The LangChain Team)
  • Intro to AI Evaluation with Langsmith (by Adrián Jiménez)
• APIs
  • langchain doc loaders
  • langchain text splitters
  • langchain SemanticChunker
  • Evaluating RAG pipelines
    • Context precision
    • Context recall
    • Faithfulness

Architecture

RAG architecture

Neo4j graph database

Crash course

Reference

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (by Patrick Lewis, etc 2020)
• Retrieval Augmented Generation (RAG) with Deep Agents | LangChain
• Retrieval Augmented Generation (RAG) with LangChain | DataCamp