Mpi One Sided Communication
MPI One-Sided Communication (RMA) is the MPI paradigm where a single process can directly read from (Get) or write to (Put) memory on a remote process without the remote process explicitly participating in the communication, enabling asynchronous data transfer patterns that can overlap computation with communication and simplify irregular communication structures.
Traditional MPI two-sided communication (Send/Recv) requires both sender and receiver to participate: the receiver must post a matching Recv before or concurrently with the sender's Send. This synchronization requirement creates challenges for irregular access patterns (where the target of each communication is data-dependent) and limits overlap opportunities.
MPI RMA Operations:
| Operation | Semantics | Use Case |
|---|---|---|
| MPI_Put | Write local data to remote window | Distributed array updates |
| MPI_Get | Read remote window data to local buffer | Irregular data gathering |
| MPI_Accumulate | Remote atomic read-modify-write | Distributed reduction |
| MPI_Get_accumulate | Atomic get + accumulate | Compare-and-swap patterns |
| MPI_Compare_and_swap | Atomic CAS on remote memory | Distributed locks |
| MPI_Fetch_and_op | Atomic fetch + operation | Counters, queues |
Window Creation: Before RMA operations, each process exposes a memory region as an MPI Window. Window types include: MPI_Win_create (existing buffer), MPI_Win_allocate (MPI allocates optimized memory), MPI_Win_allocate_shared (shared memory in same node), and MPI_Win_create_dynamic (attach/detach memory regions dynamically).
Synchronization Modes: RMA operations are non-blocking — completion must be ensured through synchronization:
- Fence synchronization: MPI_Win_fence acts as a collective barrier — all RMA ops between two fences are guaranteed complete after the second fence. Simple but synchronizes all processes.
- Post-Start-Complete-Wait (PSCW): Target process posts (MPI_Win_post), origin starts access epoch (MPI_Win_start), performs RMA operations, completes (MPI_Win_complete), target waits (MPI_Win_wait). Finer-grained than fence but requires target participation.
- Lock/Unlock: MPI_Win_lock/unlock creates passive-target access epochs — the target process does not participate at all. Supports shared locks (multiple readers) and exclusive locks (single writer). MPI_Win_lock_all provides persistent passive-target epoch for PGAS-style programming.
Performance Considerations: One-sided communication can exploit RDMA hardware (InfiniBand, iWARP) that performs remote memory access without remote CPU involvement. Key factors: latency — Put/Get can be lower latency than Send/Recv for small messages; overlap — non-blocking RMA enables computation during transfer; contention — concurrent access to same window region requires careful synchronization; progress — some MPI implementations require periodic MPI calls for background RMA progress.
Use Cases: Distributed hash tables (remote Get for lookups), stencil computations with one-sided halo exchange, distributed graph algorithms with irregular access, global arrays (GA/PGAS implemented over MPI RMA), and distributed shared-memory emulation.
MPI one-sided communication bridges the gap between message-passing and shared-memory programming models — providing the performance of RDMA-capable hardware with the portability and standardization of MPI, enabling efficient irregular communication patterns that are awkward with two-sided messaging.