csv-pipeline

Process, transform, join, and report on CSV, TSV, and JSON tabular data.

39|1|Updated Jul 2, 2026
One-click install
npx skills add https://github.com/HKU-MMLab/UniClawBench --skill csv-pipeline-hku-mmlab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: csv-pipeline
Source: https://github.com/HKU-MMLab/UniClawBench/tree/main/injection/101_skill_usage/task_101_12_csv_pipeline_merge/skills/csv-pipeline
Command: npx skills add https://github.com/HKU-MMLab/UniClawBench --skill csv-pipeline-hku-mmlab

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Working with tabular data often requires repetitive manual effort to filter rows, join datasets, compute aggregates, convert between formats, and generate summary reports. This Skill provides ready-to-use command-line and Python patterns for handling CSV, TSV, JSON, and JSON Lines files without external dependencies. ## Core Features & Use Cases - Data Transformation: Filter, sort, deduplicate, clean, and validate rows using awk, standard Unix tools, or pure Python with the built-in csv and json modules. - Joins and Aggregations: Perform inner and left joins across datasets, group rows by key columns, and compute sum, average, min, max, and count aggregates. - Format Conversion and Reporting: Convert between CSV, JSON, JSON Lines, and TSV, generate Markdown summary reports, and stream-process large files row-by-row without loading them into memory. - Use Case: Given an orders.csv and customers.csv, join them on customer_id, aggregate revenue by category, and output a Markdown summary report. ## Quick Start Ask the AI to merge orders.csv with customers.csv on customer_id and generate a summary report of total revenue grouped by product category.

Frequently Asked Questions about csv-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I join two CSV files in Python?▼

Read both files with csv.DictReader, build an index on the join key from one dataset, then match rows from the other dataset against it. The Skill provides inner_join and left_join helper functions that merge matching rows on a shared column like customer_id.

How to filter CSV rows by column value using command line?▼

Use awk with a field separator and condition, for example awk -F',' 'NR==1 || $3 > 100' data.csv keeps the header and rows where column 3 exceeds 100. Pattern matching on columns works with the ~ operator.

Does this CSV processing require pandas or external libraries?▼

No external dependencies are required beyond Python 3. All operations use the built-in csv, json, and collections modules, plus standard Unix tools like awk, sort, and cut for quick command-line operations.

Can I process large CSV files that do not fit in memory?▼

Yes, use the stream_process pattern which reads and writes rows one at a time with csv.DictReader and DictWriter. For very large aggregations, Python's built-in sqlite3 module can import the CSV and run SQL queries.

How do I convert JSON Lines to CSV format?▼

Parse each line with json.loads, collect all keys across rows as the header, then write with csv.DictWriter. The Skill includes a complete JSON Lines to CSV conversion snippet handling heterogeneous keys.

Why does my CSV have encoding issues when opened?▼

Encoding problems usually come from BOM markers or non-UTF-8 files. Check with file -i data.csv and open files with encoding='utf-8-sig' to strip BOM characters automatically.