flydsl_fused_attention_backward

Authors a fused multi-GEMM FlyDSL attention backward kernel for CDNA3/CDNA4 AMD GPUs.

178|52|Updated Jul 30, 2025
One-click install
npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fused-attention-backward-amd-agi
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: flydsl_fused_attention_backward
Source: https://github.com/AMD-AGI/GEAK/tree/main/perf_knowledge/expert_skills/skills/flydsl_fused_attention_backward
Command: npx skills add https://github.com/AMD-AGI/GEAK --skill flydsl-fused-attention-backward-amd-agi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Training and fine-tuning runs on AMD Instinct GPUs (gfx942/gfx950) often bottleneck on attention backward kernels when the vendor fast path cannot dispatch natively — for example when head dims are asymmetric (hdim_q != hdim_v) and the tuned asm/CK backward pads or falls back to a slow generic path. This Skill provides a regulated, step-by-step procedure for authoring a fused multi-GEMM FlyDSL backward kernel that avoids padding overhead and register spills. ## Core Features & Use Cases - Structural kernel design guidance: Prescribes four key decisions — fusing all GEMMs into one kernel, operand-swapped MFMA fragment orientation, KV-outer looping to avoid atomics, and wave-count selection against the 256 arch-VGPR register cap. - Gated bring-up procedure: Enforces a dependency-ordered ladder (fp32 reference, harness with ISA probe, preprocess kernel, main kernel, incremental optimization, head-to-head benchmark) with parity gates of cos >= 0.999 on every gradient. - Documented dead ends: Lists measured anti-patterns (oversized KV tiles, dQ atomic fan-out, barrier folding) so engineers do not re-derive failed optimizations. - Use Case: An MLA attention backward with HD_QK=192/HD_V=128 on gfx942 reaches 1.38x speedup per unit of useful work versus the padded-ASM baseline by following this recipe. ## Quick Start Ask the agent to author a fused FlyDSL attention backward kernel for your asymmetric-head-dim training workload on gfx942, following the gated bring-up procedure with fp32 parity validation.

Frequently Asked Questions about flydsl_fused_attention_backward

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I write a fused attention backward kernel for AMD MI300X?▼

Follow the gated procedure: build an fp32 reference with fixed causal convention, create a benchmark harness with an ISA probe, write a preprocess kernel computing D = rowsum(dO * O), then the main fused kernel with three LDS tiles and KV-outer looping. Gate each step on gradient parity (cos >= 0.999) and register counts from the ISA dump.

When should I author a custom FlyDSL attention backward kernel?▼

Author one when the vendor fast path cannot dispatch natively for your shape — typically asymmetric head dims (hdim_q != hdim_v) that force padding, or GQA/MQA variants with no tuned backward. Do not author one when a tuned asm/CK backward already dispatches natively, since that path is strong and hard to beat.

Does this kernel recipe work on gfx950 as well as gfx942?▼

The structural recipe applies to both gfx942 and gfx950, but the wave-count choice must be re-derived per generation because gfx950 has a larger register file. The measured 1.38x speedup is gfx942-specific and includes a padding tax that does not exist on gfx950.

Why is my attention backward kernel slower despite fewer instructions?▼

Instruction count reductions often lose to register spills because the fused backward carries multiple long-lived operand sets against a fixed 256 arch-VGPR cap. Check the ISA for v_accvgpr move counts — spills can relocate into the accumulator file without appearing as scratch memory.

What are the limitations of the fused backward approach?▼

It is training-only and does not apply to inference serving paths. Symmetric head dims are usually not worth authoring since the tuned asm backward dispatches natively, and non-causal attention removes the cost gradient that the longest-job-first grid ordering exploits.