skills.rest
Clay-HHK
Clay-HHKCOMMUNITY
Explore all skills by this author →
GitHub

clip

Generate image and text embeddings for zero-shot classification and cross-modal retrieval.

1|Updated Mar 5, 2026
Search Tags:#embeddings#image search#openai#content moderation#clip#vision-language#zero-shot classification
One-click install
npx skills add https://github.com/Clay-HHK/claude-skills --skill clip-clay-hhk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/Clay-HHK/claude-skills/tree/main/clip
Command: npx skills add https://github.com/Clay-HHK/claude-skills --skill clip-clay-hhk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? It enables image understanding tasks like classification, search, and content moderation without training custom models or collecting labeled datasets. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text-defined categories using OpenAI's CLIP models (RN50 through ViT-L/14) without any fine-tuning. - Semantic Image Search & Retrieval: Encode images and text into a shared embedding space for text-to-image and image-to-text retrieval, with integration examples for Chroma and FAISS vector databases. - Content Moderation: Score images against safety categories like NSFW or violent content using softmax probabilities over text prompts. - Use Case: Build a searchable photo library by encoding all images with ViT-B/32, storing normalized embeddings in a vector database, and querying with natural language like "a sunset over the ocean". ## Quick Start Use the clip skill to classify an image into custom categories and compute image-text similarity with the ViT-B/32 model.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then pass both through the model to get logits. Apply softmax to the logits to get per-label probabilities without any training.

Which CLIP model should I use: RN50, ViT-B/32, or ViT-L/14?▼

ViT-B/32 is the recommended default, balancing speed and quality at 151M parameters. RN50 is fastest, while ViT-L/14 at 428M parameters gives the best quality but is the slowest.

How do I build semantic image search with CLIP embeddings?▼

Encode all images with model.encode_image, normalize the embeddings, and store them in a vector database like Chroma or FAISS. Encode the text query the same way and rank images by cosine similarity.

Does CLIP work on CPU or does it require a GPU?▼

CLIP runs on CPU, with image encoding taking roughly 200ms per image. A GPU like a V100 reduces this to about 20ms, giving 10-50x speedup, so GPU is recommended for batch workloads.

What are the limitations of CLIP for image understanding?▼

CLIP works best for broad categories, not fine-grained tasks, and produces whole-image embeddings with no bounding boxes. It has weak spatial and counting understanding, needs descriptive text labels, and may reflect biases from its web training data.

Related Skills

Explore More Education & Research Skills→
nielpattinnielpattin

visit-webpage

Retrieve web page content, converting HTML to markdown or downloading images.

Community
Intermediate
kaminocorpkaminocorp
1

clip

Generates image and text embeddings for zero-shot classification and cross-modal retrieval.

Official
Intermediate
Orchestra-ResearchOrchestra-Research
11.5k

blip-2-vision-language

Process images for captioning, visual question answering, and image-text retrieval.

Official
Intermediate
OpenLAIROpenLAIR
1.0k

blip-2-vision-language

Bridge frozen vision encoders with large language models for multimodal reasoning.

Official
Advanced