Skills
.Work. You
Rest

event-streaming

Community

Real-time data pipelines, instantly.

Software Engineering#streaming#real-time data#kafka#data pipelines#flink#spark streaming
Authordtsong
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Skill provides expert guidance for building and optimizing real-time and near-real-time data pipelines, enabling you to process events as they happen and make data-driven decisions with minimal latency.

Core Features & Use Cases

  • Architecture Selection: Choose between batch, micro-batch, and true streaming architectures based on latency, complexity, and cost.
  • Stream Processing: Implement pipelines using Kafka, Flink, Spark Streaming, and Kafka Streams.
  • Warehouse Ingestion: Configure Snowpipe Streaming and BigQuery Storage Write API for low-latency data loading.
  • Use Case: You need to build a system that detects fraudulent credit card transactions in real-time. This Skill will help you design the architecture, choose the right stream processing framework (like Flink), configure Kafka for high throughput, and set up alerts for suspicious activity.

Quick Start

Help me choose between Flink and Spark Streaming for processing clickstream events with a 1-second latency requirement.

Dependency Matrix

Required Modules

None required

Components

references

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: event-streaming
Download link: https://github.com/dtsong/data-engineering-skills/archive/main.zip#event-streaming

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.