skills.rest
tensormux
tensormuxOFFICIAL
Explore all skills by this author →
WebsiteGitHub

Write a Triton Dequant Kernel (int4 / int8 → fp16 / bf16)

Implements Triton dequantization of int4/int8 weights to fp16/bf16 with AWQ/GPTQ/NF4 support.

54|7|Updated Apr 10, 2026
Search Tags:#triton#gptq#nf4#int8#int4#dequantization#weight-quantization
One-click install
npx skills add https://github.com/tensormux/kernel-skills --skill write-a-triton-dequant-kernel-int4-int8-fp16-bf16
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Write a Triton Dequant Kernel (int4 / int8 → fp16 / bf16)
Source: https://github.com/tensormux/kernel-skills/tree/main/skills/inference/write-triton-dequant-kernel
Command: npx skills add https://github.com/tensormux/kernel-skills --skill write-a-triton-dequant-kernel-int4-int8-fp16-bf16

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This kernel provides a structured guide to implement a Triton dequantization routine that unpacks and converts quantized int4/int8 weights into fp16 or bf16 tensors for downstream operations, enabling debugging baselines and hot-swaps of quantized weights without relying on fused GEMMs.

Core Features & Use Cases

  • Bit-unpacking and codebook support for multiple packing schemes (AWQ, GPTQ, NF4) with per-group scales and zeros.
  • Safe arithmetic and validation using fp32 intermediates to prevent overflow and enable round-trip checks against reference Python dequant implementations.
  • Use Case: debug a quantized model by exporting dequantized weights for inspection or compare against a fused dequant-GEMM path.

Quick Start

Provide an input weight tile and run the launcher to verify correct dequantization against a reference.

Frequently Asked Questions about Write a Triton Dequant Kernel (int4 / int8 → fp16 / bf16)

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I dequantize int4 and int8 weights to fp16 in Triton?▼

To dequantize int4 and int8 weights to fp16 in Triton, implement a custom kernel that unpacks bits and applies per-group scales using fp32 intermediate arithmetic to prevent overflow.

Does this Triton dequantization approach support both AWQ and GPTQ packing schemes?▼

Yes, the Triton dequantization approach supports both AWQ and GPTQ packing, handling their specific bit unpacking orders and per-group scale indexing.

Can I use a Triton kernel to unpack NF4 codebooks for weight quantization?▼

Yes, you can use a Triton kernel to unpack NF4 codebooks for weight quantization, applying optional codebook handling alongside standard int4 logic.

Why does my int4 weight dequantization overflow in Triton?▼

Int4 weight dequantization overflows in Triton when intermediate arithmetic uses lower precision, requiring fp32 intermediates to safely handle per-group scales and prevent overflow.

What is the best way to debug a quantized model without using a fused dequant GEMM?▼

The best way to debug a quantized model without a fused dequant GEMM is to export dequantized weights using a standalone Triton kernel for inspection and round-trip validation.

Related Skills

Explore More Software Engineering Skills→
ovachieverovachiever
52

tensorrt-llm

Optimize LLM inference on NVIDIA GPUs with TensorRT-LLM.

Community
Advanced
yo-stevenyo-steven

parity-testing

Verifies numerical parity for Hugging Face to Megatron weight conversions via state-dict round-trips and forward-pass tests.

Community
Intermediate
KrxGuKrxGu
54

Write an INT8 Quantized Kernel

Design INT8 quantized GEMM kernels with dp4a and INT32 accumulation for NVIDIA GPUs.

Community
Advanced
tensormuxtensormux
54

Write a Triton RMSNorm Kernel

Implement a numerically stable RMSNorm kernel in Triton with fp32 accumulation.

Official
Advanced