fiftyone-find-duplicates

What problem does it solve?

This Skill helps identify and remove near-duplicate images within FiftyOne datasets by comparing embeddings and applying similarity thresholds to group similar samples.

Core Features & Use Cases

• Near-duplicate detection: Identify groups of visually similar samples within a dataset.
• Deduplication workflow: Review results in the FiftyOne App and delete redundant samples while keeping representatives.
• Use Case: You have a large image collection and need to curate a clean dataset for model training by removing near-duplicates.

Quick Start

Launch FiftyOne, verify the brain plugin, compute embeddings for your dataset, and run the near-duplicate search to identify and review duplicates. Then proceed to deduplicate as needed.