Feature Flag

Feature Flags for ML Systems

What are Feature Flags?
Toggles that enable/disable features at runtime without deploying new code, essential for ML experimentation and gradual rollouts.

Use Cases for ML
| Use Case | Example |
|----------|---------|
| Model A/B testing | Toggle between model versions |
| Gradual rollout | Enable new model for 10% users |
| Kill switch | Disable failing model instantly |
| Experimentation | Test new prompts or parameters |

Implementation

Simple Feature Flags

import json

class FeatureFlags:
    def __init__(self, config_path):
        with open(config_path) as f:
            self.flags = json.load(f)

    def is_enabled(self, flag_name, user_id=None, default=False):
        flag = self.flags.get(flag_name)
        if not flag:
            return default

        if flag.get("enabled_for_all"):
            return True

        if user_id and flag.get("enabled_users"):
            return user_id in flag["enabled_users"]

        if flag.get("percentage"):
            return hash(user_id) % 100 < flag["percentage"]

        return flag.get("enabled", default)

LaunchDarkly/Unleash Style

from unleash_client import UnleashClient

client = UnleashClient(url="https://unleash.example.com")
client.initialize_client()

def get_model(user_context):
    if client.is_enabled("use_gpt4", context=user_context):
        return "gpt-4"
    return "gpt-3.5-turbo"

ML Experimentation

class MLExperiment:
    def __init__(self, flags):
        self.flags = flags

    def get_model_config(self, user_id):
        return {
            "model": "gpt-4" if self.flags.is_enabled("gpt4", user_id) else "gpt-3.5",
            "temperature": 0.7 if self.flags.is_enabled("high_temp", user_id) else 0.3,
            "prompt_version": self.flags.get_variant("prompt", user_id, default="v1"),
        }

Feature Flag Platforms
| Platform | Features |
|----------|----------|
| LaunchDarkly | Enterprise, ML experiments |
| Unleash | Open source |
| Split | Analytics integration |
| GrowthBook | A/B testing focus |
| ConfigCat | Simple, affordable |

Best Practices
- Use flags for all model changes
- Time-limit experiments
- Clean up old flags
- Log flag evaluations for analysis
- Use consistent hashing for user assignment