stable-diffusion-image-generation

Load a DiffusionPipeline from a pretrained model like stable-diffusion-v1-5 with torch.float16, move it to CUDA, then call it with a text prompt, num_inference_steps, and guidance_scale. The pipeline returns generated PIL images you can save directly.

SD 1.5 generates 512x512 images with lower VRAM requirements, SDXL produces higher quality 1024x1024 images, and Flux is a newer architecture with improved prompt adherence. All load through DiffusionPipeline or AutoPipelineForText2Image with the same API.

Enable model CPU offloading with pipe.enable_model_cpu_offload(), add attention slicing, and use float16 precision. For tighter budgets, use sequential CPU offload, VAE slicing, or 4-bit quantization with BitsAndBytesConfig.

Yes, load LoRA weights with pipe.load_lora_weights() and adjust strength via fuse_lora or set_adapters. Multiple LoRAs can be combined with individual adapter weights, and unloaded with unload_lora_weights.

Black images usually come from VAE dtype mismatches or the safety checker; try disabling the safety checker and keeping the VAE in float32. Noisy outputs typically mean too few inference steps or a misconfigured scheduler.

Swap the scheduler to DPMSolverMultistepScheduler and reduce steps to 20, or use LCMScheduler with the LCM LoRA for 4-step generation. Enabling xFormers memory-efficient attention and torch.compile also reduces latency.