What problem does it solve? Broken data at scale cannot be fixed row-by-row, and sending sensitive records to cloud LLMs violates PII compliance. This Skill compresses thousands of anomalous rows into a handful of semantic clusters, generates deterministic fix logic with air-gapped local models, and guarantees zero silent data loss through mathematical reconciliation. ## Core Features & Use Cases - Semantic Anomaly Compression: Embeds suspect rows with sentence-transformers and clusters them in ChromaDB or FAISS, turning 50,000 errors into roughly a dozen fixable patterns. - Air-Gapped Fix Generation: Uses Ollama-hosted models (Phi-3, Llama-3, Mistral) to output strictly validated Python lambdas, with safety gates rejecting anything containing imports, exec, eval, or os calls. - Zero-Data-Loss Reconciliation: Enforces Source_Rows == Success_Rows + Quarantine_Rows on every batch, routes low-confidence fixes to human review, and logs a full audit trail per row. - Use Case: A pipeline flags 50,000 rows with inconsistent date formats. The Skill clusters them into 12 pattern groups, generates 12 validated lambdas locally, applies them vectorized via pandas, and quarantines the 3% it cannot confidently fix. ## Quick Start Ask the agent to analyze the anomalous rows tagged NEEDS_AI from your validation layer, cluster them semantically, and generate safe local fix logic with a full audit trail.
