Prompt Tuning

Prompt Tuning and Prefix Tuning are the parameter-efficient fine-tuning methods that prepend small sequences of learnable "soft" token embeddings to the input or intermediate layers — adapting large pretrained models to downstream tasks without updating any model weights, instead learning a compact set of "virtual tokens" whose embeddings are optimized through backpropagation to steer the frozen model's behavior.

Prompt Tuning (Lester et al., 2021)

• Prepend k trainable token embeddings to input sequence.
• Frozen model: All transformer weights stay fixed.
• Only the k × d_model parameters (soft prompt) are trained.
• At inference: Soft prompt tokens + task input → model output.

class SoftPrompt(nn.Module):
    def __init__(self, n_tokens=20, d_model=1024):
        super().__init__()
        # k trainable embeddings (random or vocabulary-initialized)
        self.prompt = nn.Parameter(torch.randn(n_tokens, d_model))

    def forward(self, input_ids, model):
        input_embeds = model.embed_tokens(input_ids)  # [B, L, D]
        prompt = self.prompt.unsqueeze(0).expand(B, -1, -1)  # [B, k, D]
        full_input = torch.cat([prompt, input_embeds], dim=1)  # [B, k+L, D]
        return model(inputs_embeds=full_input)

• Key finding: At scale (> 10B params), prompt tuning matches full fine-tuning quality.
• For smaller models (< 1B), performance gap remains vs full fine-tuning.

Prefix Tuning (Li and Liang, 2021)

• Extends soft prompts to all transformer layers (not just input).
• Prepend trainable prefix to keys (K) and values (V) at every attention layer.
• Virtual tokens attend to and are attended by all real tokens.

For each layer l:
  K_l = [P_k^l ; W_k^l · x]     # prefix keys prepended
  V_l = [P_v^l ; W_v^l · x]     # prefix values prepended
  Attention uses augmented K_l, V_l → prefix influences all positions

• More expressive than input-only prompt tuning → works better for smaller models.
• Trainable parameters: 2 × num_layers × prefix_length × d_model.
• Example: GPT-2 (24 layers, d=1024), prefix=10: ~500K parameters (0.1% of model).

P-Tuning v1 and v2

• P-Tuning v1: Insert learnable tokens within input (not just at prefix) + use LSTM to generate soft token embeddings.
• P-Tuning v2: Apply prefix tuning to every transformer layer (similar to prefix tuning) → matches fine-tuning on many NLU benchmarks.

Comparison of PEFT Methods

Advantages and Limitations

• Advantages: Near-zero inference overhead (soft prompt is tiny), easy task switching (swap prompt), modular.
• Limitations: Hard to interpret (soft tokens have no human-readable meaning), less flexible than LoRA for complex adaptations, limited expressiveness at small model scales.

Applications

• Multi-task serving: One frozen model + multiple task-specific soft prompts → serve many tasks efficiently.
• Personalization: Per-user soft prompts → personalized assistant behavior without separate models.
• Continual learning: New tasks get new prompts without catastrophic forgetting of model weights.

Prompt tuning and prefix tuning are the extreme lightweight end of the parameter-efficient fine-tuning spectrum — by demonstrating that as few as 20 virtual tokens can adapt a frozen trillion-parameter model to new tasks, these methods reveal that pretrained LLMs encode broad latent capabilities that merely need steering, not retraining, offering a glimpse of a future where one set of model weights serves millions of personalized use cases through tiny learned steering vectors rather than millions of separate fine-tuned models.