tune-multiplication

What problem does it solve?

This Skill provides a compact, reproducible standard operating procedure for profiling and optimizing matrix-related operators (GEMV, GEMM, Grouped GEMM) to reach high bandwidth or TFLOPS utilization on NVIDIA Ampere/Hopper GPUs by identifying memory vs compute bottlenecks, enforcing warp-level coalescing, and selecting hardware-aligned tile and pipeline parameters.

Core Features & Use Cases

• Bottleneck classification using arithmetic intensity to decide memory-bound versus compute-bound strategies.
• Memory-bound guidance: warp coalescing rules, shared memory reuse patterns, reduce-thread sizing, and Hopper-specific tips (TMA, cp.async, L2 behavior).
• Compute-bound guidance: tensor-core alignment constraints, shared-memory budgeting, and double-buffered pipelining recommendations.
• Autotune strategy and workflow: recommended search spaces, best practices for warmup/reps, logging, validation, and comparison to cuBLAS/torch baselines.
• Case studies demonstrating real-world gains (e.g., high H200 utilization on large Llama-3 shapes) and pitfalls to avoid.
• Use case example: tune kernels for a production inference workload (large sequence GEMVs) to recover bandwidth and reduce latency.

Quick Start

Use the tune-multiplication guide to compute arithmetic intensity, verify warp-level memory coalescing, design a hardware-aligned autotune search space, and run the autotuner to record the best GEMV/GEMM configuration.