Promptable Segmentation

Promptable Segmentation is a paradigm where segmentation masks are generated based on user inputs — allowing users to interactively define what to cut out using points, bounding boxes, scribbles, or natural language, rather than relying on predefined fixed categories.

What Is Promptable Segmentation?

• Definition: Segmentation conditioned on external guidance (prompts).
• Shift: Moves from "class-based" (segment all cars) to "instance-based" (segment *this* car).
• Interaction: Often iterative; user clicks, model predicts, user corrects with more clicks.
• Flexibility: Handles objects the model has never seen before (zero-shot).

Key Prompt Types

• Spatial Prompts:
• Points: Foreground/background clicks.
• Boxes: Bounding box around the object.
• Scribbles: Rough lines drawn over the object.
• Semantic Prompts:
• Text: "Segment the red chair next to the window."
• Reference Image: "Segment objects that look like this image."

Why It Matters

• Annotation Speed: Accelerates data labeling by 10-100x.
• Usability: Makes powerful CV tools accessible to non-experts.
• Generalization: Decouples "what" to segment from "how" to segment.

Promptable Segmentation is the interface for modern computer vision — enabling dynamic human-AI collaboration for image editing, analysis, and content creation.