# NirDiamant/RAG_Techniques

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/nirdiamant-rag-techniques).**

25,455 stars · 2,985 forks · Jupyter Notebook · other

## Links

- GitHub: https://github.com/NirDiamant/RAG_Techniques
- awesome-repositories: https://awesome-repositories.com/repository/nirdiamant-rag-techniques.md

## Topics

`ai` `langchain` `llama-index` `llm` `llms` `opeani` `python` `rag` `tutorials`

## Description

This repository serves as a comprehensive knowledge base and toolkit for Retrieval-Augmented Generation (RAG). It provides a structured collection of interactive tutorials and code-based demonstrations designed to help developers optimize the accuracy and relevance of large language model responses by connecting them to external data sources.

The project distinguishes itself by offering hands-on implementations of advanced search architectures and retrieval strategies. It covers complex workflows such as multi-stage reranking, contextual compression, and self-corrective feedback loops, which are essential for reducing hallucinations and improving the precision of information retrieval. By exploring techniques like graph-structured indexing and iterative query transformation, users can move beyond basic retrieval patterns to build more robust and grounded AI systems.

The resource encompasses a wide range of practical methodologies, including hierarchical document chunking, semantic search, and various forms of query and document augmentation. These materials are organized as a series of Jupyter Notebooks, providing a clear, step-by-step learning path for engineers looking to tune system performance and master modern information retrieval patterns.

## Tags

### Artificial Intelligence & ML

- [Retrieval-Augmented Generation](https://awesome-repositories.com/f/artificial-intelligence-ml/retrieval-augmented-generation.md) — Provides foundational implementations of retrieval augmented generation patterns. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/retrieval_with_feedback_loop.py))
- [RAG Toolkits](https://awesome-repositories.com/f/artificial-intelligence-ml/rag-toolkits.md) — Provides a structured toolkit for implementing diverse document indexing and retrieval strategies.
- [Retrieval Optimization](https://awesome-repositories.com/f/artificial-intelligence-ml/retrieval-optimization.md) — Provides strategies for optimizing retrieval processes to improve the accuracy of generated responses.
- [Contextual Compression](https://awesome-repositories.com/f/artificial-intelligence-ml/contextual-compression.md) — Optimizes prompt windows by extracting only the most relevant segments from retrieved documents.
- [Performance Optimization](https://awesome-repositories.com/f/artificial-intelligence-ml/performance-optimization.md) — Implements performance tuning strategies to increase precision and reduce hallucinations in AI systems.
- [Reranking Pipelines](https://awesome-repositories.com/f/artificial-intelligence-ml/reranking-pipelines.md) — Filters and sorts initial search results using secondary models to ensure pertinent information.
- [Self-Correction Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/self-correction-architectures.md) — Demonstrates self-correcting retrieval patterns to reduce hallucinations and improve output quality. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/retrieval_with_feedback_loop.py))
- [Self-Corrective Retrieval](https://awesome-repositories.com/f/artificial-intelligence-ml/self-corrective-retrieval.md) — Evaluates retrieval quality in real-time and triggers secondary refinement steps when necessary.
- [Graph Knowledge Indexing](https://awesome-repositories.com/f/artificial-intelligence-ml/graph-knowledge-indexing.md) — Organizes data into interconnected nodes to capture complex semantic dependencies.
- [Query Transformation](https://awesome-repositories.com/f/artificial-intelligence-ml/query-transformation.md) — Refines user input through multi-step processing to improve retrieval relevance.
- [Hierarchical Chunking](https://awesome-repositories.com/f/artificial-intelligence-ml/hierarchical-chunking.md) — Segments source material into multiple granularities to balance thematic understanding with factual precision.
- [Reranking Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/reranking-strategies.md) — Implements reranking algorithms to refine the quality of retrieved context for generative models. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/HyDe_Hypothetical_Document_Embedding.py))
- [Retrieval Augmented Generation Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/retrieval-augmented-generation-techniques.md) — Implements adaptive retrieval logic to dynamically adjust search strategies. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/adaptive_retrieval.py))
- [Semantic Chunking](https://awesome-repositories.com/f/artificial-intelligence-ml/semantic-chunking.md) — Implements semantic chunking strategies to improve the relevance of retrieved document segments. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/retrieval_with_feedback_loop.py))
- [Advanced Retrieval Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/advanced-retrieval-techniques.md) — Showcases advanced retrieval techniques for optimizing information access in generative systems. ([source](https://github.com/NirDiamant/RAG_Techniques#readme))
- [Feedback Loops](https://awesome-repositories.com/f/artificial-intelligence-ml/feedback-loops.md) — Provides implementations for retrieval pipelines that incorporate feedback loops to enhance accuracy. ([source](https://github.com/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques_runnable_scripts/retrieval_with_feedback_loop.py))

### Education & Learning Resources

- [Retrieval-Augmented Generation Tutorials](https://awesome-repositories.com/f/education-learning-resources/retrieval-augmented-generation-tutorials.md) — Showcases advanced techniques for retrieval-augmented generation systems through detailed notebook tutorials. ([source](https://github.com/NirDiamant/RAG_Techniques#readme))
- [AI Engineering Tutorials](https://awesome-repositories.com/f/education-learning-resources/ai-engineering-tutorials.md) — Serves as a comprehensive knowledge base for building retrieval-augmented generation applications.
- [Machine Learning Guides](https://awesome-repositories.com/f/education-learning-resources/machine-learning-guides.md) — Offers practical examples and guides for improving language model performance.
- [Data Science Notebooks](https://awesome-repositories.com/f/education-learning-resources/data-science-notebooks.md) — Provides a collection of interactive notebooks explaining complex retrieval workflows.

### Data & Databases

- [Vector Databases](https://awesome-repositories.com/f/data-databases/vector-databases.md) — Stores document embeddings in high-dimensional space to enable rapid semantic similarity searches.
- [Knowledge Base Integrations](https://awesome-repositories.com/f/data-databases/knowledge-base-integrations.md) — Demonstrates how to integrate private knowledge bases with language models for verifiable answers.

### Software Engineering & Architecture

- [Search Architectures](https://awesome-repositories.com/f/software-engineering-architecture/search-architectures.md) — Provides architectural patterns for designing complex document indexing and retrieval workflows.