hf-cloud-serving-image-selection

Selects and verifies region-specific SageMaker serving container URIs from the AWS Deep Learning Containers catalog.

507|40|Updated Apr 26, 2026
One-click install
npx skills add https://github.com/waybarrios/opencode-power-pack --skill hf-cloud-serving-image-selection-waybarrios
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hf-cloud-serving-image-selection
Source: https://github.com/waybarrios/opencode-power-pack/tree/main/skills/hf-cloud-serving-image-selection
Command: npx skills add https://github.com/waybarrios/opencode-power-pack --skill hf-cloud-serving-image-selection-waybarrios

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve? Choosing the wrong SageMaker serving container, a stale tag, or an incompatible CUDA/AMI combination produces opaque deployment failures like "Failed to pass health check" that waste hours of debugging. This Skill removes guesswork by enforcing verified image selection from AWS's official Deep Learning Containers catalog instead of inferring URIs from memory. ## Core Features & Use Cases - Model-to-container decision rules: Maps model families (LLMs, multimodal, embeddings, rerankers, diffusion, classifiers) to the correct container family such as HuggingFace vLLM, vLLM-Omni, TEI, or DJL Inference, with HuggingFace-curated images mandatory when compatible. - Failure prevention: Documents the cu130 AMI requirement, TEI GPU/CPU variant selection, VPC/NAT image-pull constraints, and known-broken images like the NCCL-defective huggingface-pytorch-inference GPU tags. - Image mirroring script: Includes scripts/mirror_image.py to copy public ECR images into private ECR for VPC deployments without NAT gateways. - Use Case: Before deploying Qwen3-Reranker to SageMaker, preflight its config.json to discover it is a generative reranker, then select the HuggingFace vLLM DLC URI for your region instead of TEI, avoiding a 20-minute failed endpoint cycle. ## Quick Start Ask the assistant to select and verify the correct SageMaker serving container URI for your HuggingFace model and target AWS region before writing any deployment code.

Frequently Asked Questions about hf-cloud-serving-image-selection

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose a SageMaker container image for a HuggingFace model?▼

Read the URI from the AWS Deep Learning Containers catalog page and match the model family to a container: HuggingFace vLLM for text-generation LLMs, TEI for embeddings, DJL for diffusion. Substitute your region into the example URI and pass it to deploy.py --image-uri.

Should I use vLLM or TEI for a reranker model on SageMaker?▼

Check the model's config.json architectures field. Encoder cross-encoders ending in ForSequenceClassification go to TEI, while generative rerankers ending in ForCausalLM like Qwen3-Reranker require the HuggingFace vLLM DLC because TEI rejects their classifier model type.

Why does my SageMaker vLLM endpoint fail with no CloudWatch logs?▼

vLLM images tagged cu130 or higher require setting InferenceAmiVersion to al2-ami-sagemaker-inference-gpu-3-1 on the ProductionVariant. Without it the container dies on startup before creating any logs, producing a failure indistinguishable from quota or networking issues.

Can SageMaker in a VPC pull images from public ECR?▼

No, SageMaker endpoints inside a VPC without a NAT gateway cannot pull from public.ecr.aws. Use regional ECR URIs from the AWS catalog, or mirror the image to a private ECR repo with the included mirror_image.py script.

Is TGI still supported for HuggingFace LLM deployment?▼

No, Text Generation Inference is archived and models released after the archive, such as Qwen3, fail health checks on it. Use the HuggingFace vLLM DLC instead, which is also what SageMaker SDK v3 auto-routes text-generation tasks to.

Why does huggingface-pytorch-inference fail with an NCCL error on GPU?▼

Recent GPU tags ship an older NCCL than torch links against, causing ImportError: libtorch_cuda.so: undefined symbol: ncclCommResume on both g5 and g6 instances. Use DJL Inference or BYOC for GPU workloads; CPU tags are unaffected.