X-Decoder

X-Decoder is a generalized decoding framework for pixel-level vision-language tasks — capable of reducing image segmentation, image captioning, and retrieval into a unified "decode" operation using a single architecture.

What Is X-Decoder?

• Definition: A unified model for segmentation and vision-language tasks.
• Versatility: Handles generic segmentation, referring segmentation, and image captioning.
• Architecture: Hierarchical vision backbone + Text processing + Generalized decoder.
• Latent Queries: Uses two sets of queries (latent & text) to bridge pixel and semantic tasks.

Why X-Decoder Matters

• Task Unification: Does the job of multiple specialized models (segmenter + captioner).
• Granularity: Understands images at the pixel level (segmentation) and global level (captioning).
• Performance: State-of-the-art on multiple benchmarks simultaneously.

Capabilities

• Referring Segmentation: "Segment the man in the black hat."
• Image Captioning: "Describe this image." -> "A man eating a hotdog."
• Inpainting: Can support text-guided image editing workflows.

X-Decoder is a step toward general-purpose vision assistants — proving that a single model can fluently translate between pixels and natural language descriptions.