Process
Operating System Process is the fundamental unit of program execution that provides isolated memory, its own set of resources, and an independent execution context — the OS abstraction that enables multiprocessing in Python AI systems, provides crash isolation between services, and forms the basis of containerization in AI infrastructure.
What Is an OS Process?
- Definition: An instance of a running program consisting of: its own private virtual address space (memory), program counter, register state, open file handles, network connections, and at least one thread of execution — managed by the OS kernel.
- Isolation Guarantee: Process A cannot directly read or write Process B's memory — the kernel enforces virtual memory boundaries. A crash (segfault) in one process does not affect others.
- Process ID (PID): Every process has a unique integer identifier assigned by the OS. Used by ps, top, kill, and /proc/[pid]/ to monitor and manage processes.
- Creating Processes: On Unix/Linux, processes are created via fork() (copies current process) or exec() (replaces current process image with a new program). Python's subprocess and multiprocessing use these system calls.
Why Process Isolation Matters for AI Systems
- Model Serving Isolation: Running each model (embedding, reranker, LLM) as a separate process means a CUDA OOM in the LLM process cannot crash the embedding service.
- Worker Isolation in DataLoader: PyTorch DataLoader's worker processes are separate OS processes — a segfault in a preprocessing worker is caught by the DataLoader without crashing the training process.
- Container = Process: Docker containers are OS processes with namespace isolation (network, filesystem, PID) — understanding processes clarifies why containers are lightweight compared to VMs.
- Ray Actors: Ray's distributed computing abstraction maps directly to OS processes — each Ray actor is a Python process on a worker node, isolated from other actors.
- Gunicorn Workers: Production API servers (Gunicorn, uWSGI) spawn multiple worker processes — each handles requests independently, providing crash isolation and multi-core CPU utilization.
Process vs Thread Comparison
| Aspect | Process | Thread |
|---|---|---|
| Memory space | Private, isolated | Shared with parent process |
| Creation cost | High (fork ~1ms) | Low (~microseconds) |
| Memory overhead | High (full copy of address space) | Low (shared pages) |
| Crash isolation | Yes — crash doesn't affect others | No — crash kills entire process |
| Data sharing | IPC required (pipes, queues, shared memory) | Direct (but needs locks) |
| GIL | Each process has its own GIL | Shared GIL — no true parallelism for Python |
| Use case | CPU-bound parallelism | I/O-bound concurrency |
Process Life Cycle
Fork: Parent calls fork() → kernel creates identical child process (copy-on-write memory).
Exec: Child optionally calls exec() to replace itself with a new program binary.
Running: Process executes, makes system calls, uses CPU and memory.
Waiting: Process blocks on I/O, sleep, or waiting for child (wait() system call).
Zombie: Process has exited but parent has not yet called wait() to collect exit status.
Terminated: Parent called wait() — OS reclaims all resources.
IPC (Inter-Process Communication) in AI
Since processes cannot share memory directly, they communicate via IPC:
Pipes/Queues: Byte streams between processes.
from multiprocessing import Queue
q = Queue()
q.put(tensor.cpu().numpy()) # Serialize to queue
data = q.get() # Deserialize in worker
Shared Memory: Zero-copy sharing of arrays (NumPy, tensors).
from multiprocessing import shared_memory
shm = shared_memory.SharedMemory(create=True, size=array.nbytes)
# Zero-copy access from multiple processes
Sockets: TCP/UDP communication — used by Ray, gRPC, and REST APIs between services.
Memory-Mapped Files: Map a file into multiple processes' address spaces for zero-copy data access — used for large dataset sharing.
Process Management in AI Infrastructure
Supervisor / systemd: Manage long-running AI service processes — restart on crash, log output, manage environment.
Gunicorn:
gunicorn app:app --workers 4 --worker-class uvicorn.workers.UvicornWorker
Spawns 4 worker processes, each running the FastAPI/inference app — provides multi-core CPU utilization and crash isolation.
torch.multiprocessing: PyTorch's process pool with CUDA-aware shared memory — enables safe tensor sharing between training processes.
Kubernetes Pods: A pod contains one or more containers (processes) sharing a network namespace — the OS process model maps directly to Kubernetes deployment patterns.
OS processes are the fundamental isolation boundary of AI infrastructure — understanding how the kernel creates, isolates, and manages processes clarifies every aspect of container orchestration, DataLoader worker behavior, model serving architecture, and the multi-processing patterns that unlock true CPU parallelism in Python-based AI pipelines.