Glip (grounded Language-Image Pre-Training)
GLIP (Grounded Language-Image Pre-training) is a model that unifies object detection and phrase grounding — reformulating detection as a "phrase grounding" task to leverage massive amounts of image-text caption data for learning robust visual concepts.
What Is GLIP?
- Definition: Detection as grounding.
- Paradigm Shift: Instead of predicting Class ID #5, it predicts alignment with the word "cat" in the prompt.
- Data: Trained on human-annotated boxes (Gold) + Image-Caption pairs (Silver) with self-training.
- Scale: Scaled to millions of image-text pairs, far exceeding standard detection datasets.
Why GLIP Matters
- Semantic Richness: Learns attributes ("red car") and relationships, not just labels ("car").
- Data Efficiency: Utilizing caption data allows learning from the broad web.
- Zero-Shot Transfer: Performs remarkably well on benchmarks like LVIS and COCO without specific training.
How It Works
- Deep Fusion: Text and image features interact across multiple transformer layers.
- Contrastive Loss: Optimizes the alignment between region embeddings and word embeddings.
GLIP is a pioneer in vision-language unification — showing that treating object detection as a language problem unlocks massive scalability and generalization.