skills.rest
timchap
timchapCOMMUNITY
Explore all skills by this author →
GitHub

lemonade-server

Configure AMD Lemonade Server backends, ROCm channels, and speculative decoding for local LLM inference.

Updated Jul 10, 2026
Search Tags:#speculative decoding#vulkan#amd#llama.cpp#rocm#llm inference#lemonade-server
One-click install
npx skills add https://github.com/timchap/dot-hermes --skill lemonade-server-timchap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: lemonade-server
Source: https://github.com/timchap/dot-hermes/tree/main/skills/lemonade-server
Command: npx skills add https://github.com/timchap/dot-hermes --skill lemonade-server-timchap

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Running local LLM inference on AMD hardware like Strix Halo requires choosing the right backend (Vulkan, ROCm, CPU), tuning ROCm driver channels, and enabling speculative decoding — all of which involve scattered llama.cpp flags and Lemonade config keys that are easy to get wrong. ## Core Features & Use Cases - Backend Selection: Choose between Vulkan (recommended for Strix Halo gfx1151), ROCm (discrete AMD GPUs), CPU, or auto via the lemonade config CLI. - Speculative Decoding: Enable n-gram self-speculative decoding or Qwen MTP heads through --llamacpp-args for roughly 2x token throughput on unified-memory hardware. - ROCm Channel Management: Switch between stable and nightly ROCm builds and pin specific nightly binaries to capture driver-level performance gains. - Use Case: On a Ryzen AI MAX+ 395 machine with 128 GB unified memory, load Qwen3.6 35B with flash attention and MTP enabled to raise generation speed from ~40 tok/s to ~100 tok/s. ## Quick Start Ask the assistant to configure Lemonade Server to use the Vulkan backend and load your model with n-gram speculative decoding enabled.

Frequently Asked Questions about lemonade-server

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I enable speculative decoding in Lemonade Server?▼

Pass llama.cpp flags via --llamacpp-args when loading a model, for example --spec-type ngram-mod --spec-ngram-size-n 24 --draft-min 48 --draft-max 64. Native UI integration is still in draft PR #1638, so custom args are the current method.

Which backend should I use for Strix Halo, Vulkan or ROCm?▼

Vulkan is the recommended backend for Strix Halo gfx1151 because official ROCm support for that chip is incomplete. ROCm is better suited for discrete RDNA2/3/4 GPUs like the RX 6000/7000/9000 series.

Does Lemonade Server support Qwen multi-token prediction?▼

Yes, Qwen3.6 models with native MTP heads can use --flash-attn --mtp through --llamacpp-args. Benchmarks show Qwen3.6 35B-A3B reaching roughly 90-110 tok/s on Strix Halo versus a 35-45 tok/s baseline.

Why is Vulkan inference slow on my AMD system?▼

Slow Vulkan performance usually means an outdated ROCm driver stack; updating to ROCm 7.12 or newer delivered about a 40% speedup on Strix Halo. Also verify the Vulkan backend is actually active with lemonade status and try the --flash-attn flag.

When should I avoid draft-model speculative decoding on Strix Halo?▼

Draft models are generally not recommended on Strix Halo because n-gram and MTP methods give comparable or better speedups with zero extra memory cost. Draft models only make sense when a high-quality small draft model exists for your target model.

Related Skills

Explore More Software Engineering Skills→
atrawogatrawog

gpu

Monitor GPU status, VRAM, and loaded models for Ollama inference.

Community
Intermediate
FlexNetOSFlexNetOS

observability

Configure Prometheus, OpenTelemetry, and Grafana for robotics observability.

Community
Advanced
JKhyroJKhyro

llama-cpp

Run local LLM inference with GGUF models on non-NVIDIA hardware.

Community
Advanced
azaanalirazaazaanaliraza

llama-cpp

Run and quantize GGUF LLMs with llama.cpp across CPU, Apple Silicon, and GPU hardware.

Community
Advanced