Gather

MPI_Gather

MPI_Gather is a function that represents the inverse of MPI_Scatter, in the sense that one process receives elements from every process in the communicator, including itself, and gathers them into a single collection.

The function signature is as follows: int MPI_Gather(void* send_data, int send_count, MPI_Datatype send_datatype, void* recv_data, int recv_count, MPI_Datatype recv_datatype, int root, MPI_Comm communicator), where:

• send_data (↓) - represents the data sent by each process to the process with the root ID
• send_count (↓) - represents the size of the piece sent by each process (usually set as total_size / number_of_processes).
• send_datatype (↓) - the data type of the data sent by processes
• recv_data (↑) - represents the data received and stored by the root process
• recv_count (↓) - the size of the received data (usually total_size / number_of_processes)
• recv_datatype (↓) - the data type received by the root process (usually the same as send_datatype)
• root (↓) - the identifier of the process that receives the data (including itself)
• communicator (↓) - the communicator to which the processes belong (usually MPI_COMM_WORLD)

An illustration of how MPI_Gather works:

Below is an example of MPI_Scatter used together with MPI_Gather:

Example

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
 
#define ROOT 0
#define CHUNK_SIZE 5 // number of elements per process
 
int main (int argc, char **argv) {
    int rank, proc, a;
 
    int* arr;
    int* process_arr;
    int* result_arr;
 
    MPI_Init(&argc, &argv);
 
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &proc);
 
    if (rank == ROOT) {
        arr = malloc (CHUNK_SIZE * proc * sizeof(int));
        for (int i = 0; i < proc * CHUNK_SIZE; ++i) {
            arr[i] = 0;
        }
    }
 
    process_arr = malloc (CHUNK_SIZE * sizeof(int));
    MPI_Scatter(arr, CHUNK_SIZE, MPI_INT, process_arr, CHUNK_SIZE, MPI_INT, ROOT, MPI_COMM_WORLD);
 
    for (int i = 0; i < CHUNK_SIZE; i++) {
        printf("Before: rank [%d] - value = %d\n", rank, process_arr[i]);
        process_arr[i] = i;
        printf("After: rank [%d] - value = %d\n", rank, process_arr[i]);
    }
 
    if (rank == ROOT) {
        result_arr = malloc (CHUNK_SIZE * proc * sizeof(int));
    }
 
    MPI_Gather(process_arr, CHUNK_SIZE, MPI_INT, result_arr, CHUNK_SIZE, MPI_INT, ROOT, MPI_COMM_WORLD);
 
    if (rank == ROOT) {
        for (int i = 0; i < CHUNK_SIZE * proc; i++) {
            printf("%d ", result_arr[i]);
        }
        printf("\n");
    }
 
    if (rank == ROOT) {
        free(arr);
        free(result_arr);
    }
 
    free(process_arr);
 
    MPI_Finalize();
    return 0;
}