Hard Prompt Search

Hard prompt search is the process of systematically exploring the space of discrete natural language prompts to find prompt text that maximizes a language model's performance on a target task — treating the prompt as a combinatorial optimization variable rather than relying on human intuition.

Why Hard Prompt Search?

• The performance of large language models (LLMs) is highly sensitive to the exact wording, structure, and formatting of the prompt — small changes in phrasing can cause large accuracy swings.
• Human-crafted prompts may not be optimal — the prompt space is vast and unintuitive.
• Hard prompt search explores many candidate prompts automatically to find high-performing ones.

Hard Prompt Search Methods

• Paraphrase Mining: Generate paraphrases of a seed prompt using back-translation, synonym replacement, or LLM-based rewriting. Evaluate each variant on a validation set.
• Template Search: Define a prompt template with slots (e.g., "Classify the following [text type] as [label set]") and search over fill-in options.
• Evolutionary Methods: Treat prompts as individuals in a genetic algorithm — mutate (change words), crossover (combine parts of good prompts), and select (keep the best performers).
• RL-Based Search: Use reinforcement learning where the action is selecting/modifying prompt tokens and the reward is task performance.
• LLM-Guided Search: Use one LLM to generate and refine prompts for another — the "meta-prompt" approach.

Hard Prompt vs. Soft Prompt

• Hard Prompt: Actual human-readable text tokens — can be inspected, understood, and manually edited. Works with any model API (including black-box inference endpoints).
• Soft Prompt: Continuous embedding vectors prepended to the input — not human-readable, requires access to model internals.
• Hard prompt search is more practical for production deployment where models are accessed through APIs.

Hard Prompt Search Challenges

• Combinatorial Explosion: The space of possible prompts is astronomically large — exhaustive search is impossible.
• Evaluation Cost: Each candidate prompt must be evaluated on a validation set — requires many model inference calls.
• Task Specificity: Optimal prompts are highly task-specific — a prompt that works well for one task may fail on another.
• Model Specificity: Optimal prompts often differ between models — a prompt optimized for GPT-4 may not be optimal for Claude or Llama.
• Overfitting: Prompts optimized on a small validation set may not generalize to new examples.

Practical Applications

• Prompt Engineering Tools: AutoPrompt, PromptBreeder, OPRO, DSPy — frameworks that automate prompt search.
• Classification Tasks: Finding the optimal instruction and label verbalizers for text classification.
• Few-Shot Optimization: Searching for the best instruction preamble to combine with few-shot examples.

Hard prompt search transforms prompt engineering from an art into a science — replacing ad-hoc trial-and-error with systematic optimization to find the best possible prompt for any task.