skills.rest
kaminocorp
kaminocorpOFFICIAL
Explore all skills by this author →
Website𝕏@kaminocorpoGitHub

modal-serverless-gpu

Deploy and run ML workloads on Modal's serverless GPU cloud platform.

1|Updated Mar 12, 2026
Search Tags:#serverless#cloud#gpu#batch processing#modal#inference#ml deployment
One-click install
npx skills add https://github.com/kaminocorp/hermes-alpha-hunter --skill modal-serverless-gpu-kaminocorp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: modal-serverless-gpu
Source: https://github.com/kaminocorp/hermes-alpha-hunter/tree/main/skills/mlops/cloud/modal
Command: npx skills add https://github.com/kaminocorp/hermes-alpha-hunter --skill modal-serverless-gpu-kaminocorp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires modal, and includes references (resource) components.

What problem does it solve? Running GPU-intensive ML workloads typically requires provisioning servers, managing infrastructure, and paying for idle compute. This Skill provides guidance for using Modal's serverless GPU platform to run training, inference, and batch jobs on-demand with pay-per-second pricing and automatic scaling. ## Core Features & Use Cases - Serverless GPU Execution: Run Python functions on T4, A10G, L40S, A100, H100, H200, or B200 GPUs without managing infrastructure, with scale-to-zero billing. - Model Deployment as APIs: Deploy ML models as auto-scaling web endpoints using FastAPI decorators, ASGI apps, or dynamic batching for GPU-efficient inference. - Batch and Scheduled Jobs: Fan out parallel processing with .map(), persist model artifacts in Volumes, and schedule cron jobs for recurring workloads. - Use Case: Deploy a text-generation model on an A10G GPU as a REST endpoint that scales from zero to hundreds of containers automatically, paying only for actual inference time. ## Quick Start Use the modal skill to deploy my text generation model as a serverless GPU endpoint on an A10G.

Frequently Asked Questions about modal-serverless-gpu

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy an ML model as an API on Modal?▼

Define a Modal App with a function decorated with @modal.fastapi_endpoint() or @modal.asgi_app() for full FastAPI apps, then run modal deploy. Modal builds the container, provisions GPUs, and exposes an HTTPS endpoint that scales automatically.

Which GPU should I choose for inference on Modal?▼

L40S with 48GB VRAM offers the best cost-to-performance ratio for inference workloads. T4 suits budget inference of small models, while A100-80GB and H100 are better for very large models or training.

Modal vs RunPod vs Lambda Labs for GPU workloads?▼

Modal fits serverless, scale-to-zero workloads with pay-per-second billing and Python-defined infrastructure. RunPod suits longer-running pods with persistent state, Lambda Labs offers reserved instances, and SkyPilot handles multi-cloud orchestration.

How do I reduce cold start latency on Modal?▼

Increase container_idle_timeout to keep containers warm, use keep_warm=1 for always-ready containers, and load models in @modal.enter() so initialization happens once at container start rather than per request.

Why does my Modal function run out of GPU memory?▼

CUDA out-of-memory errors occur when the model exceeds GPU VRAM. Switch to a larger GPU like A100-80GB, enable gradient checkpointing, use mixed precision with torch.autocast, or enable flash attention.

How do I persist models and data between Modal runs?▼

Use modal.Volume.from_name() with create_if_missing=True and mount it into your function. Write files to the mounted path and call volume.commit() to persist changes, then volume.reload() to read the latest data.

Related Skills

Explore More Software Engineering Skills→
Ritabrata-ChakrabortyRitabrata-Chakraborty

modal

Run Python code serverlessly on cloud infrastructure with optional GPU acceleration.

Community
Advanced
ContextJet-aiContextJet-ai
29

set-up-drift-alerts

Set up drift alerts for LLM apps using OpenTelemetry GenAI semantic conventions.

Official
Intermediate
hung-phanhung-phan
1

experiment-tracking

Track machine learning experiments and manage model lifecycles with MLflow, Weights & Biases, and Trackio.

Community
Advanced
qcmuuqcmuu

skypilot-multi-cloud-orchestration

Orchestrate ML training and batch jobs across multiple cloud providers with SkyPilot.

Community
Advanced