skills.rest
80portisfound
80portisfoundCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

clip

Generate image and text embeddings for zero-shot classification and cross-modal retrieval.

Updated May 3, 2026
Search Tags:#embeddings#image search#openai#content moderation#clip#vision-language#zero-shot classification
One-click install
npx skills add https://github.com/80portisfound/vibe-learning --skill clip-80portisfound
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: clip
Source: https://github.com/80portisfound/vibe-learning/tree/main/packages/hermes/optional-skills/mlops/clip
Command: npx skills add https://github.com/80portisfound/vibe-learning --skill clip-80portisfound

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires transformers, torch, pillow, and includes references (resource) components.

What problem does it solve? It connects vision and language so you can classify images, search image collections with text queries, and moderate content without training custom models or collecting labeled data. ## Core Features & Use Cases - Zero-Shot Image Classification: Classify images into arbitrary text categories with no training data, matching ResNet-50 accuracy on ImageNet. - Semantic Image Search: Encode images and text into a shared embedding space for cross-modal retrieval and similarity matching. - Content Moderation: Score images against safety categories like NSFW or violent content using natural language labels. - Use Case: Index a folder of product photos with CLIP embeddings stored in ChromaDB, then let users search the catalog with queries like "a sunset over the ocean" and get ranked results instantly. ## Quick Start Use the clip skill to classify the attached photo against the labels dog, cat, bird, and car using the ViT-B/32 model.

Frequently Asked Questions about clip

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I do zero-shot image classification with CLIP?▼

Load a model with clip.load("ViT-B/32"), preprocess the image, tokenize your text labels, then pass both through the model to get logits. Apply softmax to the logits to get per-label probabilities without any training.

How to build semantic image search with CLIP embeddings?▼

Encode all images with model.encode_image, normalize the embeddings, and store them in a vector database like ChromaDB. Encode the text query the same way and rank images by cosine similarity against the text embedding.

CLIP vs BLIP-2 vs LLaVA for vision-language tasks?▼

CLIP is best for zero-shot classification and image-text retrieval via embeddings. BLIP-2 produces better image captions, while LLaVA supports conversational vision-language chat. CLIP does not generate text descriptions.

Which CLIP model should I use, ViT-B/32 or ViT-L/14?▼

ViT-B/32 with 151M parameters is the recommended default, balancing speed and quality. ViT-L/14 with 428M parameters gives the best accuracy but is significantly slower, so use it only when quality matters most.

What are the limitations of CLIP for image understanding?▼

CLIP works on whole images only, with no bounding boxes or object localization. It performs poorly on fine-grained categories, counting, and spatial reasoning, and can inherit biases from its web-scraped training data.

Related Skills

Explore More Software Engineering Skills→
ItsJupiter000ItsJupiter000

photo-editor

Process and enhance images with Python libraries like Pillow and OpenCV.

Community
Intermediate
ricardorochericardoroche
2

rag-design-patterns

Standardizes chunking, vector database integration, embedding management, and retrieval reranking for RAG pipelines.

Community
Intermediate
robertbr123robertbr123

segment-anything-model

Segment objects in images using the Segment Anything Model with point, box, or mask prompts.

Community
Advanced
OpenLAIROpenLAIR
1.0k

blip-2-vision-language

Bridge frozen vision encoders with large language models for multimodal reasoning.

Official
Advanced