What problem does it solve? Getting data from APIs, databases, SaaS tools, and files into a governed lakehouse usually means hand-writing ETL code, managing credentials insecurely, and babysitting deployments. This Skill guides an agent through authoring, testing, deploying, and scheduling ingestion pipelines on Tower so raw data lands reliably in Apache Iceberg tables without secrets ever touching code or chat. ## Core Features & Use Cases - Pipeline authoring with dlt: Scaffolds Tower apps that extract with dlt verified sources (Postgres, Stripe, GitHub, REST APIs) and upsert into a bronze namespace of the Tower-managed Iceberg catalog. - Secrets protocol: Enforces a workflow where credentials live in Tower secrets named per dlt's environment-variable convention, injected at runtime and never written into code, Towerfiles, or conversation. - Deploy, schedule, verify: Covers local testing with tower run --local, cloud deployment, cron schedules, retry policies, windowed backfills for large tables, and post-run verification via tower catalogs query. - Use Case: A user asks to "get our Stripe invoices into Tower daily." The agent scaffolds an ingest-stripe app, prompts the user to create the API key secret themselves, writes an idempotent upsert pipeline, deploys it, schedules a 6 AM cron, and confirms rows landed in bronze.stripe_invoices. ## Quick Start Ask the agent to build an ingestion pipeline that loads your chosen source, such as Postgres or Stripe, into the Tower lakehouse on a daily schedule.
