msmodelslim

What problem does it solve?

Huawei Ascend NPU models require efficient quantization and deployment workflows. msModelSlim provides end-to-end tools to compress, calibrate, and deploy LLMs, MoE, and multimodal models on Ascend hardware, reducing memory and latency while preserving accuracy.

Core Features & Use Cases

• Quantization types: W4A8, W8A8, W8A8C8, W8A8S, W16A16S for various trade-offs.
• Supports MindIE and vLLM-Ascend deployment pipelines.
• Calibration data preparation, precision auto-tuning, and sensitive layer analysis.
• Custom model integration via adapters, templates, and config management.
• Deployment readiness: export quantized models to vLLM-Ascend or MindIE.

Quick Start

Install msModelSlim and run a quantization for your model with the built-in config or a custom config file.