skills.rest
big4council-prog
big4council-progCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

llama-cpp

Select llama.cpp GGUF quantizations and launch commands from Hugging Face repositories.

Updated Aug 23, 2026
Search Tags:#quantization#model discovery#gguf#llama.cpp#local inference#llama-server#hugging face hub
One-click install
npx skills add https://github.com/big4council-prog/b4c-agent --skill llama-cpp-big4council-prog
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: llama-cpp
Source: https://github.com/big4council-prog/b4c-agent/tree/main/skills/mlops/inference/llama-cpp
Command: npx skills add https://github.com/big4council-prog/b4c-agent --skill llama-cpp-big4council-prog

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill solves the problem of picking the right local llama.cpp GGUF model and launching it correctly, without guesswork about quantization formats, file names, or hardware compatibility.

Core Features & Use Cases

  • Local GGUF inference & server launch: Run llama-cli or llama-server against a specific GGUF quant suitable for CPU/GPU/Metal setups.
  • Hugging Face Hub model discovery: Find llama.cpp-compatible repos, extract the repo-specific recommended quant/command from local-app pages, and confirm available GGUF files via the tree API.
  • Quant selection & file inventory: Enumerate actual .gguf filenames and sizes (including separation of projector mmproj-*.gguf files) and choose Q4/Q5/Q6/IQ variants for RAM/VRAM constraints.

Use case example: A user wants the best local model for their hardware but only knows they want “Llama 3” and have limited VRAM; the Skill discovers matching HF repos, extracts the exact recommended quant label from the ?local-app=llama.cpp view, confirms the exact .gguf file and size via the tree API, and returns a ready-to-run llama-server or llama-cli command.

Quick Start

Use the Skill to generate a llama-server command for the Hugging Face repo and recommended quant, then start the server locally.

Frequently Asked Questions about llama-cpp

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find the right GGUF quantization for llama.cpp on my hardware?▼

To run local GGUF inference using llama.cpp, you discover llama.cpp-compatible Hugging Face repositories and enumerate their actual .gguf files. The Skill then selects the correct quantization and generates a ready-to-run llama-server or llama-cli launch command tailored to your hardware.

How do I launch llama-server with a specific GGUF model from Hugging Face?▼

To launch llama-server with a specific GGUF model, you retrieve the recommended quant label from the Hugging Face local-app=llama.cpp page and confirm the exact .gguf file. This generates a ready-to-run command for CPU, Apple Silicon, CUDA, ROCm, or Intel GPU inference.

Can I run local inference on Apple Silicon or CUDA without guessing the GGUF file format?▼

Yes, you can run local inference on Apple Silicon or CUDA without guessing the GGUF file format. The process inventories actual .gguf filenames and sizes from Hugging Face repositories, separating standard model files from projector mmproj-*.gguf files for accurate hardware compatibility.

What is the best way to choose a GGUF quant under strict VRAM limits?▼

The best way to choose a GGUF quant under strict VRAM limits is to evaluate available Q4, Q5, Q6, and IQ variants from Hugging Face tree API results. By checking actual file sizes against your hardware constraints, you select the highest quality quantization that fits your memory budget for chat or technical workloads.

Does llama.cpp support multimodal projector files when discovering Hugging Face models?▼

Yes, llama.cpp supports multimodal projector files during Hugging Face model discovery. The process explicitly separates projector mmproj-*.gguf files from standard model files when inventorying repository contents via the tree API, ensuring correct file targeting for multimodal workloads.

Related Skills

Explore More Software Engineering Skills→
lxt070409lxt070409
1

llama-cpp

Run local inference with llama.cpp models and discover GGUF models from the Hugging Face Hub.

Community
Advanced
x-TheFoxx-TheFox

llama-cpp

Run local GGUF inference across CPU, Apple Silicon, and GPU backends.

Community
Intermediate
SignmanalSignmanal
1

llama-cpp

Run local GGUF model inference with llama.cpp across CPU and GPU hardware.

Community
Intermediate
ToqsickToqsick

llama-cpp

Run local GGUF inference for llama.cpp models with quantization selection and Hugging Face Hub discovery.

Community
Advanced