skills.rest
AbdullahMalik17
AbdullahMalik17COMMUNITY
Explore all skills by this author →
WebsiteGitHub

building-rag-systems

Build production RAG pipelines with semantic chunking, incremental indexing, and filtered Qdrant retrieval.

9|2|Updated Jan 31, 2026
Search Tags:#embeddings#rag#retrieval#vector search#qdrant#semantic chunking#incremental indexing
One-click install
npx skills add https://github.com/AbdullahMalik17/Hacathan_5 --skill building-rag-systems-abdullahmalik17
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: building-rag-systems
Source: https://github.com/AbdullahMalik17/Hacathan_5/tree/main/.claude/skills/building-rag-systems
Command: npx skills add https://github.com/AbdullahMalik17/Hacathan_5 --skill building-rag-systems-abdullahmalik17

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires qdrant-client, openai, pydantic, python-frontmatter, and includes scripts (resource) and references (resource) components.

What problem does it solve? Building a retrieval-augmented generation system that works in production requires solving hard problems: how to chunk documents without breaking semantic meaning, how to re-index only changed files instead of everything, and how to filter vector search results by metadata like tenant, module, or difficulty level. This Skill provides complete, working patterns for all three. ## Core Features & Use Cases - Semantic Chunking: Split Markdown documents on ## header boundaries with 400-token targets and 15% overlap, tracking prev/next chunk links for context expansion. - Incremental Indexing: Detect new, modified, and deleted files using SHA-256 content hashes stored directly in Qdrant payloads, eliminating the need for an external state database. - Filtered Retrieval: Build Qdrant filters combining tenant isolation, hardware tier ranges, module matching, and proficiency-level OR logic, with payload indexes on every filtered field. - Use Case: You are building a documentation chatbot for a multi-module course. Use this Skill to ingest Markdown lessons into Qdrant, then serve filtered semantic search through FastAPI endpoints that respect each user's tier and proficiency level. ## Quick Start Ask the AI to build a RAG ingestion pipeline that chunks your Markdown docs semantically and indexes them into Qdrant with incremental change detection.

Frequently Asked Questions about building-rag-systems

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I chunk documents for a RAG system?▼

Split documents on semantic boundaries like ## Markdown headers rather than fixed character counts. Target around 400 tokens per chunk with 10-20% overlap for context continuity, and track prev/next chunk IDs so you can expand context at retrieval time.

How to implement incremental indexing in Qdrant?▼

Store a SHA-256 file hash in each chunk's payload, then scroll Qdrant to compare indexed hashes against current filesystem hashes. This detects new, modified, and deleted files without needing an external state database, so only changed content gets re-embedded.

How do I filter Qdrant vector search by metadata?▼

Build a Filter with FieldCondition entries combined via AND logic: MatchValue for exact matches like tenant ID, Range for numeric bounds like tier levels, and MatchAny for OR logic across values. Create a payload index on every field you filter by.

What chunk size works best for RAG retrieval?▼

Around 400 tokens per chunk is a strong default based on NVIDIA benchmarks, with a 512-token maximum and 15% overlap. Merge sections smaller than 100 tokens and split oversized ones to keep chunks semantically coherent.

When should I not use this RAG approach?▼

Skip this pipeline for simple similarity search without production requirements like multi-tenancy, incremental updates, or metadata filtering. The chunking, hashing, and indexing overhead only pays off when you need filtered retrieval and ongoing content changes.

Related Skills

Explore More Software Engineering Skills→
MrOwaisAbdullahMrOwaisAbdullah
1

book-structure-generator

Generate Docusaurus book structures with chapter hierarchies and sidebars.ts configuration.

Community
Intermediate
shredbxshredbx

backend-rag-implementation

Integrate LLMs with external document sources for knowledge-grounded QA.

Community
Advanced
davicqueirozdavicqueiroz

RAG Evaluation Skill

Evaluate RAG retrieval, generation, and latency metrics against defined targets.

Community
Advanced
Prompt-or-Die-LabsPrompt-or-Die-Labs
7

Railway Qdrant Manager

Deploy and operate a Qdrant vector database on Railway for semantic search.

Official
Intermediate