stable-diffusion-image-generation

Load a DiffusionPipeline from a pretrained model like stable-diffusion-v1-5 with torch.float16, move it to CUDA, then call it with a prompt, num_inference_steps, and guidance_scale. The returned images list contains PIL images you can save directly.

SD 1.5 generates 512x512 images with lower VRAM needs, SDXL produces higher quality 1024x1024 outputs, and Flux uses newer transformer-based architectures. Diffusers auto-detects the model type when loading with DiffusionPipeline.from_pretrained.

Enable model CPU offloading with pipe.enable_model_cpu_offload(), add attention slicing and VAE slicing, or load the model in float16 precision. For large images, enable VAE tiling to decode latents in smaller chunks.

Yes, load LoRA weights with pipe.load_lora_weights() and adjust strength via fuse_lora with a lora_scale parameter. Multiple LoRAs can be combined using set_adapters with individual adapter weights.

Black images often come from the safety checker or VAE dtype mismatches, so disable the safety checker or keep the VAE in float32. Noise outputs usually mean too few inference steps or a misconfigured scheduler.

Swap to DPMSolverMultistepScheduler and reduce steps to 20, or use LCMScheduler with the LCM LoRA for 4-step generation. Enabling xFormers memory-efficient attention and torch.compile also speeds up inference.