datasphere-data-flows

What problem does it solve?

Many Datasphere projects require reliable, incremental movement and transformation of large datasets across sources and targets while preserving change data (CDC) semantics, managing costs for outbound targets, and coordinating complex multi-step orchestration; this skill provides prescriptive guidance to reduce failures, runtime inefficiencies, and manual errors across those flows.

Core Features & Use Cases

• Flow selection & design: Decision matrix for Replication Flow, Data Flow, Transformation Flow, and Task Chain selection based on CDC, real-time, batch, and transformation needs.
• Implementation guidance: Step‑by‑step creation, deployment, and monitoring instructions for replication, batch data flows with Python operators, delta-aware transformation flows, and task chain orchestration.
• Operational rules & troubleshooting: Delta prerequisites, POI licensing guidance for outbound targets, performance tuning, and failure diagnostics for common scenarios.
• Real-world example: Ingest CDS views from S/4HANA via Replication Flow with CDC into a delta-enabled local table, apply incremental SQL transformations with Transformation Flows, and orchestrate nightly jobs with Task Chains.

Quick Start

Use the datasphere-data-flows skill to create a replication flow from CDC-enabled S/4HANA CDS views, deploy to a delta-enabled local table, and add a Transformation Flow into a Task Chain for nightly incremental processing.