vector-index-tuning

Optimize vector index performance for latency, recall, and memory usage.

Updated May 20, 2026
One-click install
npx skills add https://github.com/TechCorp25/kingdom --skill vector-index-tuning-techcorp25
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vector-index-tuning
Source: https://github.com/TechCorp25/kingdom/tree/main/.claude/skills/llm-application-dev/skills/vector-index-tuning
Command: npx skills add https://github.com/TechCorp25/kingdom --skill vector-index-tuning-techcorp25

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, hnswlib, scikit-learn, qdrant-client.

What problem does it solve? Vector search systems often suffer from high latency, poor recall, or excessive memory consumption when index parameters are left at defaults. This Skill provides concrete guidance and code templates for tuning HNSW parameters, applying quantization, and monitoring search performance in production. ## Core Features & Use Cases - HNSW Parameter Tuning: Benchmark M, efConstruction, and efSearch values against recall and latency targets with ready-to-use Python templates. - Quantization Strategies: Implement INT8 scalar, product, and binary quantization to reduce memory footprint by up to 32x. - Qdrant Configuration: Create optimized Qdrant collections with preset configurations for recall, speed, balanced, or memory-optimized workloads. - Use Case: When your vector database with 10 million embeddings exceeds memory limits, use the quantization templates to compress vectors with INT8 or product quantization while maintaining 95% recall. ## Quick Start Ask the AI to recommend HNSW parameters for your dataset size, target recall, and latency budget using the vector index tuning guidance.

Frequently Asked Questions about vector-index-tuning

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I tune HNSW parameters for better recall?▼

Increase the M parameter for more graph connections and raise efSearch for deeper search traversal. Benchmark combinations of M (8-64), efConstruction (64-256), and efSearch (32-256) against ground truth to find the recall-latency tradeoff that meets your target.

What quantization method should I use for vector compression?▼

INT8 scalar quantization offers 4x compression with minimal recall loss and works well for most cases. Product quantization achieves roughly 16x compression for very large datasets, while binary quantization gives 32x compression but suits only high-dimensional embeddings.

Which vector index type works best for my dataset size?▼

Use flat exact search under 10K vectors, HNSW for 10K to 1M vectors, HNSW with quantization for 1M to 100M vectors, and IVF with product quantization or DiskANN beyond 100M vectors.

How do I configure Qdrant for low memory usage?▼

Create the collection with product quantization at 16x compression, set HNSW M to 8 with ef_construct of 64, and lower the memmap_threshold so segments move to disk sooner. This trades some recall and latency for significantly reduced RAM consumption.

Why is my vector search recall dropping in production?▼

Recall degrades from data drift, overly aggressive quantization, or efSearch set too low for your data distribution. Monitor recall continuously against ground truth, enable rescoring with oversampling for quantized searches, and reindex when data distribution shifts significantly.