@ruvector/ruvllm-cli

What problem does it solve?

This Skill simplifies running and managing Large Language Models (LLMs) locally on your own hardware, enabling powerful AI capabilities without relying on cloud services.

Core Features & Use Cases

• Local LLM Inference: Run LLMs directly on your machine with GPU acceleration (Metal/CUDA).
• Model Management: Download, list, and manage GGUF model files.
• Benchmarking: Measure the performance of your local LLM setup.
• Use Case: You need to experiment with different open-source LLMs for a specific task, but want to keep your data private and avoid API costs. This Skill allows you to download, test, and serve models locally.

Quick Start

Download the 'TheBloke/Llama-2-7B-GGUF' model with q4_k_m quantization and then start an interactive chat session.