Retry

Retry Strategies and Resilience

Why Retry?
Transient failures (network issues, rate limits, temporary overload) are common. Proper retry logic improves reliability.

Retry Patterns

Simple Retry

def simple_retry(func, max_retries=3):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            time.sleep(1)

Exponential Backoff

def exponential_backoff(func, max_retries=5, base_delay=1):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            time.sleep(delay)

With Jitter
Prevent thundering herd:

import random

def backoff_with_jitter(func, max_retries=5, base_delay=1):
    for attempt in range(max_retries):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            jitter = random.uniform(0, delay * 0.1)
            time.sleep(delay + jitter)

Tenacity Library

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=1, min=1, max=60)
)
def call_llm(prompt):
    return openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )

Retry Only Retriable Errors

from tenacity import retry, retry_if_exception_type

@retry(
    retry=retry_if_exception_type((RateLimitError, TimeoutError)),
    stop=stop_after_attempt(3)
)
def call_api():
    return api.request()

Backoff Strategies
| Strategy | Formula | Use Case |
|----------|---------|----------|
| Constant | delay | Simple cases |
| Linear | delay * attempt | Gradual increase |
| Exponential | delay * 2^attempt | Rate limits |
| Fibonacci | fib(attempt) | Moderate growth |

Best Practices
- Always include maximum retry limit
- Add jitter to prevent thundering herd
- Log retry attempts for debugging
- Use circuit breakers for sustained failures
- Only retry retriable errors
- Consider operation idempotency