skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

modal-serverless-gpu

Deploy and run ML workloads on Modal's serverless GPU cloud platform.

1|Updated Mar 5, 2026
Search Tags:#deployment#serverless#cloud#gpu#batch processing#modal#ml inference
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill modal-serverless-gpu-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: modal-serverless-gpu
Source: https://github.com/Clay-HHK/claude-skills/tree/main/modal
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill modal-serverless-gpu-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires modal, and includes references (resource) components.

What problem does it solve? Running GPU-intensive ML workloads typically requires provisioning servers, managing infrastructure, and paying for idle compute. This Skill guides you through using Modal to get on-demand GPU access, deploy models as auto-scaling APIs, and run batch jobs without managing any infrastructure. ## Core Features & Use Cases - Serverless GPU Execution: Run Python functions on T4, A10G, L40S, A100, H100, H200, or B200 GPUs with pay-per-second pricing and scale-to-zero behavior. - Model Deployment as APIs: Turn inference functions into FastAPI web endpoints, ASGI/WSGI apps, or streaming services with dynamic batching and concurrency controls. - Batch Jobs & Scheduling: Fan out parallel processing with .map(), schedule cron jobs, and persist models or data using Volumes and Secrets. - Use Case: Deploy a text-generation model on an A10G GPU as a REST endpoint that loads the model once at container start, handles concurrent requests, and scales to zero when idle. ## Quick Start Ask the AI to write a Modal app that deploys a Hugging Face text-generation model on an A10G GPU as a web endpoint, then run it with modal deploy.

Frequently Asked Questions about modal-serverless-gpu

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy a machine learning model as an API on Modal?▼

Define a Modal App with a function or class that loads your model, then add the @modal.fastapi_endpoint() or @modal.asgi_app() decorator to expose it as an HTTP endpoint. Run modal deploy script.py to make it a persistent cloud deployment with zero-downtime updates.

How do I run a Python function on a GPU with Modal?▼

Decorate a function with @app.function(gpu="A10G") specifying the GPU type, then call it with .remote() or run modal run script.py. Modal provisions the GPU container on demand and you pay per second of usage.

Modal vs RunPod vs Lambda Labs for GPU workloads?▼

Modal suits serverless, scale-to-zero workloads like APIs and batch jobs with pay-per-second billing. RunPod fits longer-running pods with persistent state, Lambda Labs fits reserved GPU instances, and SkyPilot fits multi-cloud orchestration.

Which GPUs does Modal support?▼

Modal offers T4, L4, A10G, L40S, A100 (40GB and 80GB), H100, H200, and B200 GPUs. You can request multiple GPUs with syntax like gpu="H100:4" or provide fallbacks like gpu=["H100", "A100", "L40S"].

Why are my Modal cold starts slow and how do I fix them?▼

Cold starts happen when new containers must build images and load models. Increase container_idle_timeout to keep containers warm, load models in an @modal.enter() lifecycle hook, and cache models in a Modal Volume to avoid repeated downloads.

Why does my Modal function fail with CUDA out of memory?▼

The selected GPU lacks enough VRAM for your model or batch size. Switch to a larger GPU like A100-80GB, enable gradient checkpointing, use mixed precision with torch.autocast, or reduce batch sizes.

Related Skills

Explore More Software Engineering Skills→
Orchestra-ResearchOrchestra-Research
13.0k

ray-data

Process large ML datasets with distributed streaming execution across CPU and GPU clusters.

Official
Intermediate
jvdominojvdomino

domino-model-endpoints

Deploy and monitor model endpoints in Domino with Grafana and NVIDIA Triton support.

Community
Advanced
hung-phanhung-phan
1

experiment-tracking

Track machine learning experiments and manage model lifecycles with MLflow, Weights & Biases, and Trackio.

Community
Advanced
qcmuuqcmuu

skypilot-multi-cloud-orchestration

Orchestrate ML training and batch jobs across multiple cloud providers with SkyPilot.

Community
Advanced