Skills
.Work. You
Rest

implement-partitioned-connector

Official

Implement partitioned connectors

Software Engineering#streaming#data ingestion#partitioning#batch#connector#spark
Authordatabrickslabs
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill automates the implementation of complex data connectors that support both batch and streaming partitioned reads, ensuring efficient data ingestion from various sources.

Core Features & Use Cases

  • Partitioned Reads: Implements SupportsPartition for batch and SupportsPartitionedStream for streaming partitioned data.
  • Interface Conformance: Ensures connectors adhere to LakeflowConnect and partitioning interfaces.
  • Use Case: Develop a streaming connector for a time-series database where data is naturally partitioned by time ranges, allowing Spark to efficiently process new data in micro-batches.

Quick Start

Implement a partitioned connector for the 'my_source' data source.

Dependency Matrix

Required Modules

None required

Components

references

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: implement-partitioned-connector
Download link: https://github.com/databrickslabs/lakeflow-community-connectors/archive/main.zip#implement-partitioned-connector

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.