dataset-manager
OfficialManage TPC-H datasets
Software Engineering#benchmarking#parquet#data optimization#dataset management#gpu acceleration#tpch
Authorsirius-db
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill streamlines the management of TPC-H datasets in Parquet format, enabling efficient generation, optimization, and inspection for database benchmarking.
Core Features & Use Cases
- Data Generation: Create TPC-H datasets at any scale factor using an optimized Rust-based generator.
- Parquet Optimization: Consolidate and rewrite Parquet files for improved performance, automatically leveraging GPU (cudf) or falling back to CPU (pyarrow).
- Dataset Inspection: Analyze the structure, row group sizes, and file sizes of your datasets.
- Use Case: Optimize a large TPC-H SF100 dataset to reduce file sizes and improve query performance on Sirius, ensuring efficient benchmarking.
Quick Start
Generate TPC-H data for scale factor 100 in the default output directory.
Dependency Matrix
Required Modules
None requiredComponents
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: dataset-manager Download link: https://github.com/sirius-db/sirius/archive/main.zip#dataset-manager Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.