Mpi Derived Datatype

MPI Derived Datatypes are the user-defined data layout descriptors that allow MPI to send and receive non-contiguous or heterogeneous data in a single communication operation — eliminating the need to pack scattered data into contiguous buffers before sending, which reduces memory copies, simplifies code, and enables MPI to optimize network transfers of complex data structures like matrix subblocks, struct arrays, and irregular grid regions directly from application memory.

Why Derived Datatypes

• Basic MPI_Send: Sends contiguous buffer of identical elements.
• Real data is often non-contiguous: Column of a row-major matrix, struct fields, subarray.
• Without derived types: Manual pack → send → unpack. Error-prone, wastes memory.
• With derived types: MPI handles data layout → send directly from original data structure.

Core Derived Type Constructors

Example: Sending a Matrix Column

// Matrix: double A[100][100] (row-major)
// Send column 5: A[0][5], A[1][5], ..., A[99][5]
// These are 100 elements, each 100 doubles apart

MPI_Datatype col_type;
MPI_Type_vector(
    100,           // count: 100 blocks
    1,             // blocklength: 1 element per block
    100,           // stride: 100 elements between blocks
    MPI_DOUBLE,    // base type
    &col_type
);
MPI_Type_commit(&col_type);

MPI_Send(&A[0][5], 1, col_type, dest, tag, comm);
MPI_Type_free(&col_type);

Example: Sending a C Struct

typedef struct {
    int id;
    double position[3];
    char label[8];
} Particle;

MPI_Datatype particle_type;
int blocklengths[] = {1, 3, 8};
MPI_Aint displacements[3];
MPI_Datatype types[] = {MPI_INT, MPI_DOUBLE, MPI_CHAR};

Particle p;
MPI_Get_address(&p.id, &displacements[0]);
MPI_Get_address(&p.position, &displacements[1]);
MPI_Get_address(&p.label, &displacements[2]);
// Convert to relative offsets
for (int i = 2; i >= 0; i--)
    displacements[i] -= displacements[0];

MPI_Type_create_struct(3, blocklengths, displacements, types, &particle_type);
MPI_Type_commit(&particle_type);

// Now send array of particles directly
Particle particles[1000];
MPI_Send(particles, 1000, particle_type, dest, tag, comm);

Subarray Type (Domain Decomposition)

// Global grid: 1000 × 1000
// Local subdomain: rows 250-499, cols 250-499 (250×250)
int sizes[] = {1000, 1000};       // global dimensions
int subsizes[] = {250, 250};      // subdomain size
int starts[] = {250, 250};        // starting indices

MPI_Datatype subarray;
MPI_Type_create_subarray(2, sizes, subsizes, starts,
                         MPI_ORDER_C, MPI_DOUBLE, &subarray);
MPI_Type_commit(&subarray);

Performance Considerations

• MPI internally handles non-contiguous packing → often uses optimized memcpy.
• RDMA-capable networks (InfiniBand): Can send non-contiguous data without CPU packing.
• Very complex types: May fall back to element-by-element copy → profile to verify.
• Rule of thumb: Derived types are always cleaner code; usually equal or better performance than manual pack.

MPI derived datatypes are the expressiveness layer that makes MPI practical for real scientific computing — by describing arbitrarily complex data layouts in a portable, type-safe manner, derived datatypes allow domain scientists to focus on physics and algorithms rather than low-level data marshaling, while enabling MPI implementations to optimize network transfers based on the actual memory layout.