Helm

Helm Charts for ML Deployments

What is Helm?
Package manager for Kubernetes, using charts (templates) to deploy applications with configurable values.

Basic Helm Chart Structure

Chart.yaml

apiVersion: v2
name: llm-inference
description: LLM inference server
version: 1.0.0
appVersion: "1.0.0"

values.yaml

replicaCount: 2

image:
  repository: llm-inference
  tag: "v1.0.0"
  pullPolicy: IfNotPresent

model:
  name: "gpt-4"
  maxTokens: 4096

resources:
  limits:
    nvidia.com/gpu: 1
    memory: 16Gi
  requests:
    nvidia.com/gpu: 1
    memory: 8Gi

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetGPUUtilization: 70

Deployment Template

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}-llm
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    spec:
      containers:
      - name: llm-server
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        resources:
          {{- toYaml .Values.resources | nindent 10 }}
        env:
        - name: MODEL_NAME
          value: {{ .Values.model.name }}
        - name: MAX_TOKENS
          value: "{{ .Values.model.maxTokens }}"

Install and Upgrade

# Install
helm install llm-prod ./llm-inference -f values-prod.yaml

# Upgrade
helm upgrade llm-prod ./llm-inference -f values-prod.yaml

# Rollback
helm rollback llm-prod 1

# Uninstall
helm uninstall llm-prod

Popular ML Helm Charts
| Chart | Purpose |
|-------|---------|
| vLLM | High-throughput inference |
| text-generation-inference | HuggingFace TGI |
| ray-cluster | Distributed training |
| mlflow | Experiment tracking |
| triton-inference-server | NVIDIA serving |

Best Practices
- Use values files per environment
- Version your charts
- Test templates with helm template
- Use helm secrets for sensitive values
- Keep charts in git with application code