skills.rest
aminojagh
aminojaghCOMMUNITY
Explore all skills by this author →
GitHub

new-endpoint

Adds a new REST API endpoint resource to an existing dlt pipeline.

Updated Jun 15, 2026
Search Tags:#data pipeline#data ingestion#etl#rest api#endpoint#pagination#dlt
One-click install
npx skills add https://github.com/aminojagh/LLMZC --skill new-endpoint-aminojagh
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: new-endpoint
Source: https://github.com/aminojagh/LLMZC/tree/main/05_02_dlt_workshop/.claude/skills/new-endpoint
Command: npx skills add https://github.com/aminojagh/LLMZC --skill new-endpoint-aminojagh

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Extending an existing dlt REST API pipeline with a new endpoint requires understanding the current source configuration, researching the API's pagination and response structure, and matching existing patterns like incremental loading and write dispositions. This Skill guides that process end to end so the new resource integrates consistently. ## Core Features & Use Cases - Declarative resource addition: Adds endpoints to the RESTAPIConfig resources list with correct paths, params, data selectors, pagination, and response_actions for optional endpoints. - Custom resource fallback: Implements custom @dlt.resource functions with RESTClient for date-iterated endpoints or non-standard pagination that declarative config cannot express. - Isolated testing and consistency review: Tests the new resource with with_resources() and add_limit(), then checks alignment with existing incremental, merge, and processing-step patterns. - Use Case: You have a working dlt pipeline pulling GitHub issues and want to also pull repository members. The Skill reads the pipeline, researches the members endpoint, adds the resource with 404-ignore response actions, and verifies the load. ## Quick Start Ask the AI to add a new endpoint such as "user profiles" to your existing dlt pipeline and let it read the pipeline file, research the API, and implement the resource.

Frequently Asked Questions about new-endpoint

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new endpoint to an existing dlt pipeline?▼

Read the pipeline's @dlt.source function and RESTAPIConfig, research the new endpoint's path, params, and pagination, then add it to the resources list. Test it in isolation using source.with_resources("new_resource").add_limit(1) before running the full pipeline.

When should I use a custom @dlt.resource instead of RESTAPIConfig?▼

Use a custom @dlt.resource when the endpoint cannot be described declaratively, such as date-iterated endpoints requiring a loop over a date range or non-standard pagination. Use RESTClient from dlt.sources.helpers.rest_client for authenticated HTTP calls inside the custom resource.

How do I handle optional endpoints that return 404 in dlt?▼

Use response_actions in the resource config with {"status_code": 404, "action": "ignore"} so missing child resources are skipped silently. You can also match response body content or pass a callable to patch malformed responses before dlt parses them.

Can dlt fetch child resources concurrently?▼

Yes, set "parallelized": True on a child resource to fetch records for all parents concurrently. Be aware that all child pages for one parent are buffered in memory before yielding, so avoid it for parents with very large child sets.

How do I test a single new dlt resource without reloading everything?▼

Call pipeline.run(source.with_resources("new_resource").add_limit(1)) to load only the new resource with limited pages. You can also iterate source.resources["my_resource"] directly to debug pagination without writing to the destination.

Related Skills

Explore More Data & Analytics Skills→
untitled-data-companyuntitled-data-company
20

dlt-skill

Create and maintain dlt pipelines from APIs, databases, and other sources.

Official
Advanced
scumunnascumunna
1

pipeline-orchestration-and-data-quality

Orchestrate marketing data pipelines with idempotent loads and data quality checks.

Community
Advanced
dlt-hubdlt-hub
53

debug-pipeline

Inspect dlt pipeline traces and load packages to diagnose post-execution errors.

Official
Intermediate
dlt-hubdlt-hub
53

new-endpoint

Add a REST API endpoint to an existing dlt pipeline configuration.

Official
Intermediate