skills.rest
aminojagh
aminojaghCOMMUNITY
Explore all skills by this author →
GitHub

find-source

Finds the best dlt source for a given API or data provider.

Updated Jun 15, 2026
Search Tags:#pipeline#data ingestion#rest api#connector#dlt#source discovery#dlthub
One-click install
npx skills add https://github.com/aminojagh/LLMZC --skill find-source-aminojagh
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: find-source
Source: https://github.com/aminojagh/LLMZC/tree/main/05_02_dlt_workshop/.claude/skills/find-source
Command: npx skills add https://github.com/aminojagh/LLMZC --skill find-source-aminojagh

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When building a data pipeline with dlt, the first challenge is figuring out whether a ready-made connector exists for your data source or whether you need to build a custom REST API pipeline. This Skill removes that guesswork by systematically searching verified sources, dlthub context, and the web to identify the right ingestion path. ## Core Features & Use Cases - Source Classification: Maps requests like databases, files, or REST APIs to the correct dlt core source (sql_database, filesystem, rest_api). - Verified Source Lookup: Runs dlthub pipeline init listing and searches dlthub context to find pre-built, maintained connectors. - Web Validation: Confirms findings against authoritative API documentation and avoids third-party integrators. - Use Case: A user asks to ingest data from Alpaca Markets. The Skill checks verified sources, validates the API endpoints via web search, presents viable options, and asks the user to pick a single endpoint before handing off to pipeline creation. ## Quick Start Ask the assistant to find a dlt source for the API or data provider you want to ingest, for example by saying find a dlt source for stripe payments data.

Frequently Asked Questions about find-source

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a dlt source for a specific API?▼

Provide the source name and the data you want to ingest. The workflow classifies the request, lists verified sources with dlthub pipeline init, searches dlthub context, and validates findings against official API documentation.

What dlt core sources exist for databases and files?▼

dlt provides sql_database for databases like Postgres, MySQL, and Oracle, filesystem for CSV, Parquet, JSONL, and cloud storage like S3 or GCS, and rest_api for HTTP-based services. Matching requests are routed to these core sources directly.

When should I use a verified dlt source instead of building a custom pipeline?▼

A verified source is almost always preferable because it is pre-built and maintained. If one exists for your provider, run dlthub pipeline init with that source and destination rather than building a REST API pipeline from scratch.

Can dlt handle GraphQL or non-REST data sources?▼

The REST API pipeline workflow handles REST APIs and sometimes GraphQL. Non-REST core sources like sql_database or filesystem are handed off to their dedicated toolkits or a general coding session instead.

Why does the source search avoid third-party API providers?▼

Third-party integrators and proxies add unnecessary dependencies and may be less reliable. The workflow prefers authoritative sources, such as the provider's official API documentation, to ensure accurate endpoint and access information.

Related Skills

Explore More Data & Analytics Skills→
dlt-hubdlt-hub
53

find-source

Identify dlt source connectors for APIs and data providers.

Official
Intermediate
towertower
19

create-rest-api-pipeline

Scaffold a deterministic dlt REST API pipeline for Tower apps.

Official
Advanced
TwuanMinnTwuanMinn

api-security-best-practices

Implement authentication, authorization, input validation, rate limiting, and TLS for API endpoints.

Community
Advanced
aporto-techaporto-tech

Scrape

Extract text and structured data from web pages using browser automation.

Official
Intermediate