Skills
.Work. You
Rest

spark-python-data-source

Community

Connect Spark to any external system.

Software Engineering#streaming#etl#batch#connector#spark#pyspark#data source
AuthorAradhya0510
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill enables developers to build custom Python data sources for Apache Spark, allowing seamless integration with external systems that lack native connectors.

Core Features & Use Cases

  • Custom Connectors: Develop batch and streaming readers/writers for databases, APIs, message queues, or custom protocols.
  • Data Integration: Pull data from or push data to external systems using Spark DataFrames.
  • Use Case: Connect Spark to a legacy REST API to ingest real-time data into a Delta Lake table, or build a connector to write Spark DataFrame results to a proprietary data store.

Quick Start

Use the spark-python-data-source skill to create a batch reader for a custom REST API.

Dependency Matrix

Required Modules

None required

Components

references

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: spark-python-data-source
Download link: https://github.com/Aradhya0510/databricks-cv-accelerator/archive/main.zip#spark-python-data-source

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.