Argmax Flows

Argmax Flows is a generative model for discrete data that defines a continuous-time flow in a continuous latent space and maps to discrete outputs using the argmax operation — the model generates continuous vectors and converts them to discrete tokens by taking the argmax over category dimensions.

Argmax Flow Approach

• Continuous Latent: Define a flow or diffusion process in a continuous latent space (one dimension per category).
• Argmax Mapping: Map continuous vectors to discrete tokens: $x_{discrete} = \text{argmax}(z)$ over the category dimension.
• Dequantization: Inverse direction: add continuous noise within each discrete category cell — enable continuous density estimation.
• Exact Likelihood: Unlike discrete diffusion, argmax flows can provide exact log-likelihood bounds.

Why It Matters

• Principled: Provides a theoretically clean bridge between continuous generative models and discrete data.
• Density Estimation: Enables exact likelihood computation for discrete data — useful for evaluation and comparison.
• Alternative: Offers a different approach to discrete generation than discrete diffusion or autoregressive models.

Argmax Flows are continuous flows with discrete outputs — mapping continuous generative processes to discrete tokens through the argmax operation.