vllm-omni-perf

What problem does it solve?

vLLM-Omni performance tuning helps engineers identify and reduce bottlenecks across autoregressive and diffusion pipelines, enabling faster inference, lower latency, and better resource utilization.

Core Features & Use Cases

• Benchmarking suite for end-to-end latency and throughput across models and hardware.
• Optimization levers including TeaCache, Cache-DiT, quantization, CPU offloading, and parallelism tuning.
• Use Case: A data science team benchmarks a DiT-based diffusion model before and after applying optimizations to quantify speedups.

Quick Start

Run a baseline benchmark on a sample model to establish a performance floor.