skills.rest
truefoundry
truefoundryOFFICIAL
Explore all skills by this author →
Website𝕏@truefoundryGitHub

truefoundry-llm-deploy

Deploy LLM models on TrueFoundry with YAML manifests and the tfy CLI.

1|Updated Mar 27, 2026
Search Tags:#manifest#huggingface#vllm#nim#tgi#gpu-inference#llm-deploy
One-click install
npx skills add https://github.com/truefoundry/tfy-deploy-skills --skill truefoundry-llm-deploy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: truefoundry-llm-deploy
Source: https://github.com/truefoundry/tfy-deploy-skills/tree/main/skills/llm-deploy
Command: npx skills add https://github.com/truefoundry/tfy-deploy-skills --skill truefoundry-llm-deploy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

Deploys LLM models and GPU-accelerated inference servers on TrueFoundry using YAML manifests and the tfy CLI or REST API.

Core Features & Use Cases

  • Production-grade LLM deployment with vLLM, TGI, or NVIDIA NIM
  • GPU provisioning, model caching, and health probes for reliable serving
  • Use cases include hosting Gemma, Llama, Mistral, or HuggingFace models for inference

Quick Start

Provide the HuggingFace model ID and workspace to generate a ready-to-deploy manifest.

Frequently Asked Questions about truefoundry-llm-deploy

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I deploy a HuggingFace model for GPU inference on TrueFoundry?▼

You can deploy a HuggingFace model for GPU inference on TrueFoundry by providing the HuggingFace model ID and workspace to generate a ready-to-deploy YAML manifest using the tfy CLI or REST API.

Which inference servers are supported for LLM deployment with TrueFoundry?▼

TrueFoundry supports production-grade LLM deployment and GPU-accelerated inference using vLLM, TGI (Text Generation Inference), or NVIDIA NIM servers.

Can I host Llama, Mistral, or Gemma models using vLLM on TrueFoundry?▼

Yes, you can host model families such as Llama, Mistral, and Gemma using vLLM on TrueFoundry, with built-in GPU provisioning, model caching, and health probes for reliable serving.

What is the best way to ensure reliable LLM serving with health probes on TrueFoundry?▼

The best way to ensure reliable LLM serving is to use TrueFoundry's manifest generation, which enforces prerequisites, artifact handling, and robust health probes for production-grade GPU deployments.

Do I need YAML manifests and the tfy CLI to deploy LLMs on TrueFoundry?▼

Yes, you need YAML manifests and the tfy CLI or REST API to deploy LLMs on TrueFoundry, which handles GPU provisioning, model caching, and artifact management for production-grade inference.

Related Skills

Explore More Software Engineering Skills→
truefoundrytruefoundry
13

llm-deploy

Deploy LLMs and ML inference servers on TrueFoundry with GPU-accelerated endpoints.

Official
Advanced
tomztomz

coreweave-inference

Deploy scalable LLM inference endpoints on Kubernetes with vLLM, TGI, or Triton.

Community
Advanced
Clouder0Clouder0

build

Generate and validate reproducible build configurations for multi-language repositories.

Community
Basic
rish2jainrish2jain

nvidia-nim

Deploy and operate NVIDIA Inference Microservices for GPU-accelerated AI inference.

Community
Advanced