Open-Vocabulary Detection

Open-Vocabulary Detection (OVD) is an object detection paradigm where models can locate and classify arbitrary objects — described by free-form text, rather than being limited to a fixed list of base categories (like the 80 classes in COCO).

What Is Open-Vocabulary Detection?

• Definition: Detecting objects described by any text prompt.
• Core Idea: Replace the final classification layer (logits) with a text-image alignment score (dot product).
• Training: Uses base classes with bounding boxes + image-text pairs (captions) for vocabulary expansion.
• Inference: User provides a list of potentially novel class names -> Model finds them.

Why OVD Matters

• Flexibility: Detect "pokemon", "r2d2", or "covid mask" without retraining.
• Language Integration: Bridges the gap between NLP and Computer Vision.
• Search: Enables powerful semantic search loopups in video/image databases.

Key Models

• ViLD: Vision and Language knowledge Distillation.
• GLIP: Grounded Language-Image Pre-training.
• OWL-ViT: Open-World Localization Vision Transformer.
• Grounding DINO: State-of-the-art open-set detection.

Open-Vocabulary Detection is bringing search-like flexibility to vision — allowing us to find anything we can name, essentially "Googling" the physical world.