database-maintenance

Plan and execute lakehouse compaction, snapshot expiration, and relational index maintenance with FinOps cost tracking.

Updated Feb 10, 2026
One-click install
npx skills add https://github.com/vesviet/agent-skills --skill database-maintenance-vesviet
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: database-maintenance
Source: https://github.com/vesviet/agent-skills/tree/main/core/skills/security-data/database-maintenance
Command: npx skills add https://github.com/vesviet/agent-skills --skill database-maintenance-vesviet

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Operational data stores degrade over time: streaming writes create thousands of small files in Iceberg and Delta tables, unmanaged snapshots explode metadata, and fragmented B-tree or pgvector indexes slow queries and inflate cloud bills. This Skill provides structured runbooks and guardrails to perform maintenance safely without locking production tables or deleting in-flight data. ## Core Features & Use Cases - Lakehouse Compaction & Cleanup: Bin-pack compaction merging Deletion Vectors into 256MB Parquet blocks, manifest rewriting, 7-day snapshot expiration with a 50-snapshot floor, and orphan vacuuming with a 72-hour grace window. - Relational & Vector Index Maintenance: Concurrent reindexing of HNSW/IVFFlat pgvector indexes, bloat detection, VACUUM ANALYZE, and mandatory 2-second lock timeouts for safe DDL. - Data FinOps Attribution: Calculates file reduction ratio, storage reclaimed, and projected S3 GET savings, tagging runs with CostCenter, Environment, and TableOwner. - Use Case: A data platform engineer notices query latency rising on a streaming Iceberg table. The Skill guides them through assessing small-file distribution, running rewrite_data_files, expiring stale snapshots, and emitting a FinOps report showing reclaimed storage and cost savings. ## Quick Start Use the database-maintenance skill to plan a compaction and snapshot expiration run for my Iceberg table lakehouse.silver.orders with a rollback plan and FinOps report.

Frequently Asked Questions about database-maintenance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compact small files in Apache Iceberg tables?▼

Run bin-pack compaction via CALL system.rewrite_data_files with a target file size of 256MB (min 64MB, max 512MB). In Iceberg v3 this also merges Deletion Vectors into rewritten base Parquet blocks, eliminating read amplification.

How to safely expire Iceberg snapshots without breaking time travel?▼

Use expire_snapshots with a 7-day TTL while retaining a minimum of 50 recent snapshots via retain_last. This safety floor protects rollback capability and downstream batch pipelines that reference recent snapshots.

Can I rebuild pgvector HNSW indexes without locking tables?▼

Yes, use REINDEX INDEX CONCURRENTLY for HNSW and IVFFlat indexes so reads and writes continue during the rebuild. Always set lock_timeout to 2 seconds first to abort immediately if the lock cannot be acquired.

Why does orphan file vacuuming delete active data files?▼

Vacuuming without a grace window can remove files written by concurrent in-flight transactions. Enforce a mandatory 72-hour grace period in remove_orphan_files so only files older than three days are purged.

How do I prevent S3 503 Slow Down errors on Iceberg writes?▼

Enable object storage prefix hashing by setting write.object-storage.enabled = true on the Iceberg table. This distributes objects across S3 prefixes and eliminates throttling from high-throughput streaming writes to a single prefix.

What FinOps metrics should database maintenance report?▼

Track file reduction ratio, storage reclaimed in GB, and projected monthly S3 GET request savings. Tag every maintenance run with CostCenter, Environment, and TableOwner metadata for cost attribution.