python-data-io

Defines Python conventions for reading and writing parquet, CSV, Excel, and database data.

Updated Jul 29, 2026
One-click install
npx skills add https://github.com/chris-prener/dev-kit --skill python-data-io-chris-prener
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: python-data-io
Source: https://github.com/chris-prener/dev-kit/tree/main/dev-kit/skills/python-data-io
Command: npx skills add https://github.com/chris-prener/dev-kit --skill python-data-io-chris-prener

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Python projects often accumulate inconsistent data I/O code: hardcoded paths, dtype guessing on CSV reads, unparameterized SQL, and credentials scattered through source files. This Skill establishes a single set of conventions for file-based I/O, database connections, path handling, and credential management so every data read/write follows the same safe patterns. ## Core Features & Use Cases - Format selection guidance: Decision table for choosing parquet (polars), CSV, Excel, JSON, or pickle based on data shape and audience, with parquet as the default for tabular data. - Database connectivity: Parameterized SQLAlchemy queries with connection pooling, plus DuckDB for querying parquet files directly without loading them into memory. - Safe path and credential handling: Enforces pathlib.Path over string concatenation and environment-variable credentials loaded via python-dotenv, never hardcoded secrets. - Use Case: You need to read a folder of partitioned parquet files, filter to one store, and write the result to PostgreSQL. The Skill provides the lazy polars scan with predicate pushdown, the SQLAlchemy engine pattern, and the write_database call in one consistent style. ## Quick Start Ask the AI to write Python code that reads a parquet file with polars, filters it, and writes the result to a database using environment-variable credentials.

Frequently Asked Questions about python-data-io

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I read and write parquet files in Python with polars?▼

Use pl.read_parquet() to read and df.write_parquet() to write. For large files, use pl.scan_parquet() with filters and column selection so only needed data is loaded, then call .collect() to materialize the result.

Should I use polars or pandas for tabular data in Python?▼

Prefer polars for new code: it is faster, has stricter null typing, and its lazy API mirrors dplyr-style pipelines. Pandas remains acceptable where an existing codebase or a dependency such as certain plotting or ML libraries requires it.

How do I connect Python to PostgreSQL with SQLAlchemy safely?▼

Create one module-level engine with create_engine and credentials from environment variables, then use parameterized queries with text() and :param placeholders. Never f-string user input into SQL, and dispose the engine when finished.

Can DuckDB query parquet files without loading them into Python?▼

Yes. DuckDB queries parquet files directly with read_parquet('data/*.parquet') inside SQL, including glob patterns across files. Call .pl() on the result to get a polars DataFrame or .df() for pandas.

Why should CSV reads specify a schema in production Python code?▼

Explicit schema_overrides prevent dtype guessing, which can silently change column types when data shifts between runs. Schema inference with infer_schema_length=None is acceptable only for exploratory work, never in shipped code.

When is this data I/O skill not the right tool?▼

It does not cover validating data after reading, orchestrating I/O steps in pipelines, or installing I/O packages. Those belong to separate data validation, pipeline orchestration, and dependency management workflows.