clip

Generates image and text embeddings for zero-shot classification and cross-modal retrieval.

1|Updated Mar 12, 2026
One-click install
npx skills add https://github.com/kaminocorp/hermes-alpha-hunter --skill clip-kaminocorp
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/kaminocorp/hermes-alpha-hunter/tree/main/skills/mlops/models/clip
Command: npx skills add https://github.com/kaminocorp/hermes-alpha-hunter --skill clip-kaminocorp

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? Classifying images, searching image collections by text, or moderating visual content normally requires training custom models on labeled data. CLIP removes that requirement by matching images and text in a shared embedding space, enabling zero-shot vision tasks with plain natural language labels. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text categories without any training data or fine-tuning. - Semantic Image Search: Index image embeddings and retrieve matches using natural language queries, with optional integration into vector databases like Chroma or FAISS. - Content Moderation: Score images against safety categories such as NSFW or violent content using softmax probabilities. - Use Case: Given a folder of product photos, encode them once with ViT-B/32, then let users search the catalog with queries like "a red sneaker on white background" and return ranked results. ## Quick Start Use the CLIP skill to classify the attached image against the labels dog, cat, bird, and car and show the confidence scores.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then pass both through the model to get logits. Apply softmax to the logits to get per-label probabilities without any training.

How to build semantic image search with CLIP embeddings?▼

Encode all images with model.encode_image, normalize the embeddings, and store them. Encode a text query the same way, then rank images by cosine similarity. You can store embeddings in Chroma or FAISS for larger collections.

CLIP vs BLIP-2 vs LLaVA for vision-language tasks?▼

CLIP is best for zero-shot classification, retrieval, and similarity scoring. BLIP-2 produces better image captions, while LLaVA supports conversational vision-language chat. CLIP does not generate text descriptions.

Which CLIP model should I use: RN50, ViT-B/32, or ViT-L/14?▼

ViT-B/32 is the recommended default, balancing speed and quality at 151M parameters. RN50 is fastest, while ViT-L/14 gives the best quality at 428M parameters but runs significantly slower.

Does CLIP require a GPU for image encoding?▼

CLIP runs on CPU but a GPU is 10-50x faster. Image encoding takes roughly 200ms on CPU versus 20ms on a V100 GPU, so GPU is recommended for batch processing or search indexing.

What are the limitations of CLIP for image understanding?▼

CLIP works best for broad categories, not fine-grained distinctions. It produces no bounding boxes, has weak spatial and counting understanding, needs descriptive text labels, and can inherit biases from its web training data.