Zero-Copy Memory

Zero-copy memory is the memory access model where GPU reads host-resident data directly without explicit copy into device memory - it removes setup copy overhead but pays higher per-access latency compared with local VRAM.

What Is Zero-copy memory?

• Definition: Mapped host memory accessed by GPU through interconnect address translation.
• Benefit: No explicit memcpy stage before kernel launch for suitable workloads.
• Latency Tradeoff: Remote host access is slower than accessing local global memory on device.
• Best Fit: Useful for read-once, low-reuse, or integrated-memory scenarios.

Why Zero-copy memory Matters

• Simplified Flow: Can reduce programming overhead for streaming or sparse host-resident inputs.
• Startup Speed: Avoids copy setup cost for small or transient data payloads.
• Memory Flexibility: Allows processing of data not fully staged in VRAM.
• Edge Use Cases: Practical in embedded systems with tighter shared-memory integration.
• Prototype Utility: Helpful for quick experiments before full transfer optimization.

How It Is Used in Practice

• Access Profiling: Measure whether zero-copy latency is acceptable for target kernel pattern.
• Pattern Restriction: Use zero-copy for low-reuse paths and avoid repeated random access hot loops.
• Hybrid Strategy: Combine zero-copy for cold paths with explicit copies for high-reuse tensors.

Zero-copy memory is a niche but useful data-access strategy for specific GPU workflows - it trades lower setup overhead for higher access latency, so workload fit is critical.