pytorch-quantization

What problem does it solve?

This Skill addresses the need to reduce model size and accelerate inference for PyTorch models, particularly for deployment on resource-constrained environments.

Core Features & Use Cases

• INT8 Quantization: Convert floating-point models to use 8-bit integers for reduced memory footprint and faster computation.
• Post-Training Quantization (PTQ): Apply quantization to an already trained model without retraining.
• Quantization Aware Training (QAT): Train a model with quantization simulation to maintain higher accuracy.
• Use Case: Deploy a computer vision model on an edge device where memory and processing power are limited, significantly improving its performance and reducing power consumption.

Quick Start

Use the pytorch-quantization skill to demonstrate manual tensor quantization and load a pre-quantized ResNet50 model.