lambda-labs-gpu-cloud

Install lambda-cloud-client, configure it with your API key, and call launch_instance with a region, instance type name, and SSH key names. The instance typically becomes active in 3-15 minutes depending on GPU count.

For 7B models, a single A100 40GB works well; for 70B models, use an 8x H100 instance. A10 or A6000 instances suit inference and development, while B200 offers maximum training performance.

Lambda Labs provides dedicated instances with pre-installed Lambda Stack, persistent filesystems, and no egress fees. RunPod offers cheaper spot and serverless options, while Vast.ai is a marketplace with the lowest prices but less consistency.

Only data on attached Lambda filesystems persists; the root volume at /home/ubuntu is ephemeral and lost on termination. Filesystems must be attached at launch time and mounted at /lambda/nfs/<name>.

Connection refused usually means the instance is still initializing; single-GPU instances take 3-5 minutes and multi-GPU up to 15 minutes. Also verify the instance status is active and your SSH key was added before launch.

Use 1-Click Clusters, which provide 16-512 H100 or B200 GPUs with Slurm and InfiniBand preconfigured. Submit jobs with srun and torchrun, or manually coordinate torchrun across instances using private IPs and a shared master address.