effect-stream

Build pull-based streaming pipelines with Effect Stream, Sink, and Channel in TypeScript.

3|Updated Apr 1, 2026
One-click install
npx skills add https://github.com/mpsuesser/opencode-effect-enforcer --skill effect-stream-mpsuesser
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: effect-stream
Source: https://github.com/mpsuesser/opencode-effect-enforcer/tree/main/skills/effect-stream
Command: npx skills add https://github.com/mpsuesser/opencode-effect-enforcer --skill effect-stream-mpsuesser

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires effect, @effect/platform-node.

What problem does it solve? Writing correct streaming code in TypeScript requires handling backpressure, resource cleanup, concurrency, and encoding errors, and developers often misuse Effect Stream APIs or miss edge cases like split UTF-8 characters and unbounded collection. ## Core Features & Use Cases - Stream Creation and Transformation: Construct streams from iterables, paginated APIs, DOM events, Node.js readables, and async iterables, then transform them with map, filter, flatMap, grouping, debounce, and throttle. - Encoding and Decoding: Pipe streams through NDJSON and SchemaBinary codec channels for wire-format serialization with schema validation and error recovery. - Concurrency and Resource Safety: Merge, broadcast, and share streams with controlled fan-out, and manage resources with scoped acquisition and forkScoped consumers. - Use Case: Consume a paginated REST API with Stream.paginate, enrich each item concurrently with mapEffect, filter invalid entries, and write batches to a database with runForEach. ## Quick Start Ask the agent to build an Effect Stream pipeline that fetches all pages from a paginated API, transforms the results concurrently, and decodes NDJSON responses with schema validation.

Frequently Asked Questions about effect-stream

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a stream from a paginated API in Effect?▼

Use Stream.paginate with an initial cursor and an effectful function returning the current page plus Option.some(nextCursor) or Option.none() to stop. The stream pulls pages lazily with backpressure until the cursor is exhausted.

How do I decode NDJSON with schema validation in Effect Stream?▼

Pipe the raw stream through Ndjson.decodeSchemaString(MySchema)() using Stream.pipeThroughChannel. For binary sources like TCP sockets, use Ndjson.decode() which accepts Uint8Array chunks, and catch NdjsonError with Stream.catchTag for recovery.

What is the difference between Stream.broadcast and broadcastN?▼

Stream.broadcast starts the producer immediately, so late subscribers miss earlier values unless replay is configured. Stream.broadcastN waits until all n downstream streams subscribe before starting, guaranteeing every consumer sees the full sequence.

Does Stream.retry resume where the stream failed?▼

No, Stream.retry restarts the entire stream from the beginning on each retry according to the given Schedule. Schedule errors are also unioned into the stream's error channel and must be handled explicitly.

When should I avoid Stream.runCollect?▼

Avoid runCollect on unbounded production event streams since it accumulates all elements in memory. Use runForEach or runDrain for side-effecting consumers, and reserve runCollect for tests with take(n) on finite streams.