📚 RAG Explorer

A hands-on learning project that lets you compare four RAG strategies side-by-side using your own PDFs and locally-running Ollama models — no API keys required.

---

🎯 What This Project Is

I built this to deeply understand how RAG works in practice — not just read about it. The app lets you upload any PDFs, then query them using four increasingly sophisticated RAG pipelines while seeing exactly what each strategy does differently.

What you can experiment with:

• Switch between 4 RAG strategies on the same documents
• Choose from multiple Ollama LLMs and embedding models
• Control chunking strategy, text splitter, chunk size, overlap, and top-K
• Pull missing Ollama models directly from the UI
• See agent reasoning traces (Agentic RAG) and rewritten queries (Advanced RAG)

---

🧠 What is RAG?

RAG (Retrieval-Augmented Generation) is a technique that grounds an LLM's answers in external documents rather than relying solely on its training data.

                    ┌─────────────────────────────────────┐
  Your PDF ──────►  │  1. CHUNK  →  2. EMBED  →  3. STORE │  (Indexing)
                    └─────────────────────────────────────┘
                                                    │
  Your Question ──► 4. EMBED QUERY                 │
                    5. SIMILARITY SEARCH ◄──────────┘
                    6. RETRIEVE top-K chunks
                    7. LLM generates answer from chunks

Without RAG, an LLM can only answer from its training data (potentially outdated, and it can hallucinate). With RAG, the LLM answers from your documents.

---

🏗️ The Four RAG Strategies

1. 🔍 Naive RAG

The baseline. Simple and fast.

Query → Embed → Cosine Similarity Search → Top-K Chunks → LLM → Answer

When to use: Quick prototypes, simple Q&A, when your documents are clean and well-structured.

Limitation: No query understanding, retrieves whatever is most similar even if redundant.

---

2. ⚡ Advanced RAG

Three enhancements on top of Naive RAG:

Query
  │
  ▼
[Query Rewriting]  ← LLM rewrites the query to be retrieval-friendly
  │
  ▼
[MMR Retrieval]    ← Maximal Marginal Relevance: diverse, non-redundant chunks
  │
  ▼
[Contextual Compression] ← LLM strips irrelevant sentences from each chunk
  │
  ▼
LLM → Answer

MMR explained: Instead of picking the 4 most similar chunks (which might all say the same thing), MMR picks chunks that are both relevant and different from each other.

When to use: Production systems, when answer quality matters more than speed.

---

3. 🕸️ GraphRAG

Builds a knowledge graph during indexing. Enables multi-hop reasoning.

INDEXING PHASE:
  Each chunk → LLM extracts entities + relationships → NetworkX graph
  "Paris" ──[capital of]──► "France"
  "France" ──[in]──► "Europe"

QUERY PHASE:
  Query → Extract query entities → Graph traversal (1-2 hops) → Relevant chunks

Why this matters: Classic RAG can answer "What is the capital of France?" but struggles with "What continent contains the capital of France?" — a 2-hop question. GraphRAG can follow entity links to answer it.

Limitation: Slow indexing (one LLM call per chunk to extract entities). Quality depends on how well the LLM extracts structured data.

---

4. 🤖 Agentic RAG

An LLM agent that decides when, how many times, and what to retrieve.

User Question
    │
    ▼
┌─────────────────────────────────┐
│  AGENT (ReAct loop)             │
│                                 │
│  Think: "I need to find X"      │
│    │                            │
│    ▼                            │
│  [search_documents tool]  ───► retrieve chunks
│    │                            │
│    ▼                            │
│  Think: "I also need Y"         │
│    │                            │
│    ▼                            │
│  [search_documents tool]  ───► retrieve more chunks
│    │                            │
│    ▼                            │
│  "I have enough context now"    │
└─────────────────────────────────┘
    │
    ▼
Final Answer (with Agent Trace shown in UI)

When to use: Complex multi-part questions, when the question structure isn't known in advance, research-style queries.

---

🛠️ Tech Stack

Component	Library	Purpose
UI	Streamlit 1.58	Web interface
LLM + Embeddings	Ollama + LangChain-Ollama	Local inference
Document Loading	LangChain Community (PyPDF)	PDF parsing
Vector Store	FAISS	Similarity search
Text Splitting	LangChain Text Splitters	Chunking
Graph	NetworkX	Knowledge graph (GraphRAG)
Agent	LangChain Agents	ReAct agent (Agentic RAG)

---

📋 Prerequisites

Requirement	Version	Install
Python	3.11+	python.org
Ollama	latest	ollama.com
Git	any	git-scm.com

macOS: brew install python ollama Linux: Follow ollama.com/download Windows: Use WSL2 + the Linux instructions

---

🚀 Setup — Step by Step

1. Clone the repository

git clone https://github.com/harshb/rag-learning.git
cd rag-learning

2. Create a virtual environment

python -m venv .venv

# Activate it:
source .venv/bin/activate        # macOS / Linux
.venv\Scripts\activate           # Windows

3. Install Python dependencies

pip install -r requirements.txt

4. Start Ollama

ollama serve
# Leave this running in a separate terminal

5. Pull the required models

You need at least one LLM and one embedding model.

Recommended minimum setup:

# Embedding model (required to index PDFs)
ollama pull nomic-embed-text

# LLM (choose one)
ollama pull llama3.2          # fast, good quality (~2 GB)
ollama pull qwen2.5-coder:7b  # great at structured reasoning (~4 GB)
ollama pull mistral           # well-rounded (~4 GB)

Or pull from the UI: The app shows ✅ / ⚠ next to each model dropdown and has a ⬇ Pull button that downloads the model with a live progress bar.

6. Run the app

streamlit run app.py

Open http://localhost:8501 in your browser.

---

🎮 How to Use

1. Upload PDFs using the sidebar file uploader
2. Select a RAG strategy using the segmented control at the top
3. Configure models — choose your LLM and embedding model (check they show ✅)
4. Tune chunking — adjust splitter, strategy, chunk size, and overlap
5. Click ⚙ Process PDFs — this indexes your documents
6. Ask questions in the chat box at the bottom

Tips:

• Start with Naive RAG to establish a baseline, then switch strategies to compare
• For GraphRAG, indexing is slow (LLM call per chunk) — use smaller PDFs first
• Agentic RAG shows an Agent Trace so you can see exactly what it searched for
• Advanced RAG shows the rewritten query it used for retrieval

---

⚙️ Configuration Reference

Setting	What it does
LLM Model	The model used for answering questions (and query rewriting / entity extraction)
Embedding Model	Converts text to vectors for similarity search — must match what you used when indexing
Text Splitter	Algorithm for splitting text: Recursive (default), Character, or Token-based
Chunking Strategy	Fixed size, Sliding Window (high overlap), Sentence-aware, or Paragraph-aware
Chunk Size	Max characters per chunk. Smaller = more precise retrieval. Larger = more context per chunk.
Overlap	How much consecutive chunks share. Prevents facts from being split across chunk boundaries.
Top K	How many chunks to retrieve. More = more context, but also more noise.

Chunking Strategies Explained

Fixed Size:      [chunk1][chunk2][chunk3]          no overlap, clean boundaries
Sliding Window:  [chunk1     ][chunk2     ]         each chunk overlaps the previous by ≥50%
Sentence-aware:  [sentence. sentence.][sentence.]  splits at sentence endings
Paragraph-aware: [paragraph]   [paragraph]         splits at double newlines

---

📁 Project Structure

rag-learning/
├── app.py              # Main Streamlit application
├── requirements.txt    # Python dependencies
├── README.md           # This file
└── data/               # (optional) Put sample PDFs here

---

🐛 Troubleshooting

Connection refused when processing PDFs → Ollama isn't running. Run ollama serve in a separate terminal.

model "xyz" not found → Pull the model first: ollama pull xyz or use the ⬇ Pull button in the UI.

GraphRAG takes too long → It runs one LLM call per chunk (capped at 40 chunks). Use smaller PDFs or a faster model like phi3.

Agentic RAG gives no answer → Not all Ollama models support tool calling. Use llama3.2, qwen2.5-coder:7b, or mistral.

The app is slow → LLMs run on CPU by default. If you have an Apple Silicon Mac, Ollama automatically uses the Metal GPU. For NVIDIA GPUs on Linux, follow Ollama GPU setup.

---

🎯 The Sweet Spot — What Actually Moves the Needle

Chunking Strategy, Embedding Model, Retrieval Strategy, and Metadata Strategy usually impact answer quality more than the LLM itself.

Swapping GPT-4 for GPT-3.5 on a well-tuned pipeline often hurts less than using the wrong chunk size. Here's what I learned by experimenting with real documents like the Indian Constitution.

---

1. Chunk Size

The problem with too-small chunks (chunk_size = 100):

Article 21 Right to Life        ← Chunk 1
and Personal Liberty...         ← Chunk 2  (retriever might only get this)
No person shall be deprived...  ← Chunk 3

Context breaks. The retriever may return Chunk 2 without Chunk 1, producing an incomplete answer.

The problem with too-large chunks (chunk_size = 3000):

Article 20 + Article 21 + Article 22 + Article 23  ← all in one chunk

Retrieval becomes imprecise — the chunk is so broad it matches everything and nothing.

Sweet spot by document type:

Document Type	Recommended Chunk Size
FAQ / Short answers	300 – 500
General PDFs	500 – 1000
Legal documents	800 – 1500
Books / Long narrative	1000 – 2000
Source code	300 – 800
Indian Constitution	800 – 1200

---

2. Chunk Overlap

No overlap (overlap = 0) — information at chunk boundaries gets lost:

Chunk 1: chars 0–1000     → Article 21 starts here...
Chunk 2: chars 1000–2000  → Article 21 continues...   ← connection severed

With overlap (chunk_size = 1000, overlap = 200):

Chunk 1: chars    0–1000   ┐
Chunk 2: chars  800–1800   ┤ last 200 chars of Chunk 1 repeated at the start of Chunk 2
Chunk 3: chars 1600–2600   ┘

The boundary context is preserved. The cost is slightly more chunks, more storage, and more embeddings to compute.

Typical overlap ratios:

Chunk Size	Recommended Overlap
500	50 – 100
1000	100 – 200
2000	200 – 400

---

3. Top-K Retrieval

How many chunks to hand to the LLM after retrieval.

Too low (top_k = 1): Fast and cheap, but a single chunk rarely contains the full answer. For "Can police arrest me without reason?", you need at least Article 21, Article 22, and the detention clauses — none of which sit in the same chunk.

Too high (top_k = 20): Good recall, but you flood the LLM context with noise. The model starts averaging over irrelevant chunks.

Top-K by use case:

Use Case	Top K
FAQ bot	3
PDF chat	4 – 6
Legal documents	5 – 10
Research / Agentic RAG	10 – 20

For the Constitution: start at 5, increase to 8–10 for complex multi-article questions.

---

4. Text Splitter Types

Splitter	How it works	Good for
CharacterTextSplitter	Splits every N characters, ignores structure	Quick tests only
RecursiveCharacterTextSplitter	Tries \n\n → \n → → "" in order, preserving semantic units	General purpose (default)
TokenTextSplitter	Counts LLM tokens, not characters — more accurate for model context limits	Any LLM-specific pipeline
MarkdownHeaderTextSplitter	Splits on #, ##, ### headings	Documentation, wikis
HTMLHeaderTextSplitter	Splits on <h1>, <h2> tags	Web content
SemanticChunker	Embeds sentences, splits where meaning changes (cosine distance)	Advanced — expensive but powerful

Best splitter for the Indian Constitution:
Not Recursive, not Character. An article-aware splitter that treats each Article as its own chunk gives dramatically better results:

❌ RecursiveCharacter:    "...and personal liberty. No person shall be deprived of his li"
                         "fe or personal liberty except according to procedure established..."

✅ Article-based:        "Article 21: Right to Life and Personal Liberty.
                          No person shall be deprived of his life or personal liberty
                          except according to procedure established by law."

The RecursiveCharacterTextSplitter with separators=["Article ", "\n\n", "\n", " "] gets close to this.

---

5. Embedding Models

The embedding model converts text into a vector. Better models = more semantically accurate similarity search = better chunks retrieved.

Model	Dimensions	Speed	Quality	Best for
all-MiniLM-L6-v2	384	⚡⚡⚡	⭐⭐	Learning, prototypes
nomic-embed-text	768	⚡⚡	⭐⭐⭐	Local Ollama (great balance)
mxbai-embed-large	1024	⚡	⭐⭐⭐⭐	Higher quality local
bge-base-en-v1.5	768	⚡⚡	⭐⭐⭐⭐	Production retrieval
bge-large-en-v1.5	1024	⚡	⭐⭐⭐⭐⭐	Best local quality
text-embedding-3-small	1536	API	⭐⭐⭐⭐	OpenAI (production)
voyage-3	1024	API	⭐⭐⭐⭐⭐	State of the art retrieval

Why it matters in practice:

Question: "What is Article 21?"

Model: all-MiniLM-L6-v2   → similarity score = 0.71  → Article 22 retrieved (wrong)
Model: nomic-embed-text    → similarity score = 0.86  → Article 21 retrieved (correct)

The LLM is identical. The answer quality is completely different.

Rule: Never change the embedding model after indexing. If you switch models, you must re-process all documents — the old vectors become meaningless.

---

6. Putting It Together — Production Architecture

If I were building a production Constitution AI today:

PDFs
  │
  ▼
Article-aware splitter
  chunk_size:    1200
  chunk_overlap: 150
  │
  ▼
Embedding: nomic-embed-text  (or bge-base-en-v1.5 for higher quality)
  │
  ▼
Vector Store: FAISS (local) / OpenSearch (production)
  │
  ▼
Retrieval: MMR, top_k = 5, fetch_k = 15
  │
  ▼
Reranking: Cross-encoder reranker (cuts noise from top 15 → best 5)
  │
  ▼
Context Compression: LLMChainExtractor (strips irrelevant sentences)
  │
  ▼
LLM: llama3.1 / mistral (local)  or  GPT-4o (production)
  │
  ▼
Answer + Sources

---

7. The Mental Model

Chunking    = How information is stored
Embeddings  = How meaning is represented
Retriever   = How information is found
Prompt      = How information is presented to the LLM
LLM         = How information is explained to the user

Optimise in that order. A bad chunking strategy cannot be fixed by a better LLM.

---

📖 Key Observations

Chunking beats strategy switching.
A well-tuned chunk size often beats switching from Naive → Advanced RAG. Tune your chunking first.

Embedding model ≠ LLM.
They do entirely different jobs. You can pair a tiny, fast embedding model with a large, slow LLM — they don't need to match.

MMR is quietly one of the best improvements.
Maximum Marginal Relevance (Advanced RAG) silently fixes redundant retrieval — getting 4 chunks that all say the same thing. Force diversity.

Agents are powerful but unpredictable.
Agentic RAG can loop and self-correct, but the same question may take different retrieval paths on each run. Don't use it where determinism matters.

GraphRAG has a narrow sweet spot.
Only worth the slow indexing for highly interconnected knowledge bases — legal documents, scientific literature, or wikis. Overkill for most PDF Q&A.

---

🤝 Contributing

Pull requests are welcome! If you want to extend this:

• Add a new RAG strategy — add it to RAG_TYPES, RAG_INFO, and implement *_retrieve() + wire it into run_rag()
• Add a new Ollama model — append to LLM_MODELS or EMBEDDING_MODELS
• Improve GraphRAG — better entity extraction prompt, smarter graph traversal
• Add evaluation — implement RAGAS metrics to quantitatively compare strategies

---

📜 License

MIT — use it, fork it, learn from it.

---

Built as a learning project to deeply understand RAG systems by building them from scratch.