MPI C - 將2d陣列段收集到一個全局陣列中

Question

我想從我的主進程中接收其所有其他進程的所有組件後，打印一個動態分配的2d陣列。我所說的組件是指子數組或塊。

我已經使代碼通用於進程的數量。下圖將幫助您瞭解塊在整個陣列中的排列方式。每個塊由一個進程處理。只是雖然這裏，讓我們假設我運行使用12道工序（本身我有8個內核）的程序，使用下面的命令：

mpiexec -n 12 ./gather2dArray 

這是圖，它專門針對12個流程場景：

喬納森在這個question的答案幫了我很多，但不幸的是我還沒有能夠完全實現我想要的。

我首先創建塊到每個進程中，我將它們命名爲grid。每個數組都是一個動態分配的二維數組。我還創建了全局數組（universe），僅通過master進程（＃0）可見。

最後，我必須使用MPI_Gatherv(...)將所有子數組組合到全局數組中。然後我繼續顯示本地數組和全局數組。

當我用上面的命令運行程序時，我得到分段錯誤，當我達到MPI_Gatherv(...)函數。我無法弄清楚我做錯了什麼。我已經提供了以下完整代碼（大量註釋）：

編輯

我有固定的代碼中的一些是非。現在MPI_Gatherv()有些成功。我能夠正確打印整個全局數組的第一行（我檢查過程的各個元素，並且它們總是匹配）。但是當我到達第二排時，出現了一些象形文字，最後出現了分段錯誤。我一直無法弄清楚那裏有什麼問題。仍在研究它..

#include <stdio.h> 
#include <stdlib.h> 
#include <mpi.h> 
#include <time.h> 

void print2dCharArray(char** array, int rows, int columns); 


int main(int argc, char** argv) 
{ 
    int master = 0, np, rank; 
    char version[10]; 
    char processorName[20]; 
    int strLen[10]; 

    // Initialize MPI environment 
    MPI_Init(&argc, &argv); 

    MPI_Comm_size(MPI_COMM_WORLD, &np); 
    if (np != 12) { MPI_Abort(MPI_COMM_WORLD,1); } 
    MPI_Comm_rank(MPI_COMM_WORLD, &rank); 

    // We need a different seed for each process 
    srand(time(0)^(rank * 33/4)); 

    int nDims = 2;    // array dimensions 
    int rows = 4, columns = 6; // rows and columns of each block 
    int prows = 3, pcolumns = 4; // rows and columns of blocks. Each block is handled by 1 process 

    char** grid = malloc(rows * sizeof(char*)); 
    for (int i = 0; i < rows; i++) 
    grid[i] = malloc(columns * sizeof(char)); 

    char** universe = NULL;   // Global array 
    char* recvPtr;     // Pointer to start of Global array 
    int Rows = rows * prows;   // Global array rows 
    int Columns = columns * pcolumns; // Global array columns 
    int sizes[2];      // No of elements in each dimension of the whole array 
    int subSizes[2];     // No of elements in each dimension of the subarray 
    int startCoords[2];    // Starting coordinates of each subarray 
    MPI_Datatype recvBlock, recvMagicBlock; 

    if (rank == master){   // For the master's eyes only 
    universe = malloc(Rows * sizeof(char*)); 
    for (int i = 0; i < Rows; i++) 
     universe[i] = malloc(Columns * sizeof(char)); 

    // Create a subarray (a rectangular block) datatype from a regular, 2d array 
    sizes[0] = Rows; 
    sizes[1] = Columns; 
    subSizes[0] = rows; 
    subSizes[1] = columns; 
    startCoords[0] = 0; 
    startCoords[1] = 0; 

    MPI_Type_create_subarray(nDims, sizes, subSizes, startCoords, MPI_ORDER_C, MPI_CHAR, &recvBlock); 

    // Now modify the newly created datatype to fit our needs, by specifying 
    // (lower bound remains the same = 0) 
    // - new extent 
    // The new region/block will now "change" sooner, as soon as we reach a region of elements 
    //   occupied by a new block, ie. every: (columns) * sizeof(elementType) = 
    MPI_Type_create_resized(recvBlock, 0, columns * sizeof(char), &recvMagicBlock); 

    MPI_Type_commit(&recvMagicBlock); 
    recvPtr = &universe[0][0]; 
    } 

    // populate arrays 
    for (int y = 0; y < rows; y++){ 
    for (int x = 0; x < columns; x++){ 
     if (((double) rand()/RAND_MAX) <= density) 
    grid[y][x] = '#'; 
     else 
     grid[y][x] = '.'; 
    } 
    } 


    // display local array 
    for (int i = 0; i < np; i++){ 
    if (i == rank) { 
     printf("\n[Rank] of [total]: No%d of %d\n", rank, np); 
     print2dCharArray(grid, rows, columns); 
    } 
    MPI_Barrier(MPI_COMM_WORLD); 
    } 


    /* MPI_Gathering.. */ 
    int recvCounts[np], displacements[np]; 

    // recvCounts: how many chunks of data each process has -- in units of blocks here -- 
    for (int i = 0; i < np; i++) 
    recvCounts[i] = 1; 

    // prows * pcolumns = np 
    // displacements: displacement relative to global buffer (universe) at which to place the 
    //        incoming data block from process i -- in block extents! -- 
    int index = 0; 
    for (int p_row = 0; p_row < prows; p_row++) 
    for (int p_column = 0; p_column < pcolumns; p_column++) 
     displacements[index++] = p_column + p_row * (rows * pcolumns); 

    // MPI_Gatherv(...) is a collective routine 
    // Gather the local arrays to the global array in the master process 
    // send type: MPI_CHAR  (a char) 
    // recv type: recvMagicBlock (a block) 
    MPI_Gatherv(&grid[0][0], rows * columns, MPI_CHAR, //: parameters relevant to sender 
      recvPtr, recvCounts, displacements, recvMagicBlock, master, //: parameters relevant to receiver 
      MPI_COMM_WORLD); 

    // display global array 
    MPI_Barrier(MPI_COMM_WORLD); 
    if (rank == master){ 
    printf("\n---Global Array---\n"); 
    print2dCharArray(universe, Rows, Columns); 
    } 

    MPI_Finalize(); 
    return 0; 
} 


void print2dCharArray(char** array, int rows, int columns) 
{ 
    int i, j; 
    for (i = 0; i < rows; i++){ 
    for (j = 0; j < columns; j++){ 
     printf("%c ", array[i][j]); 
    } 
    printf("\n"); 
    } 
    fflush(stdout); 
} 

以下是我得到的輸出。無論我嘗試什麼，我都無法克服這一點。正如您所看到的，使用4個進程的前4個塊可以正確打印全局數組的第一行。當跳到下一行我們得到象形文字..

hostname@User:~/mpi$ mpiexec -n 12 ./gather2darray 
MPICH Version: 3User 
Processor name: User 

[Rank] of [total]: No0 of 12 
. . # . . # 
# . # # # . 
. . . # # . 
. . # . . . 

[Rank] of [total]: No1 of 12 
. . # # . . 
. . . . # # 
. # . . # . 
. . # . . . 

[Rank] of [total]: No2 of 12 
. # # # . # 
. # . . . . 
# # # . . . 
. . . # # . 

[Rank] of [total]: No3 of 12 
. . # # # # 
. . # # . . 
# . # . # . 
. . . # . . 

[Rank] of [total]: No4 of 12 
. # . . . # 
# . # . # . 
# . . . . . 
# . . . . . 

[Rank] of [total]: No5 of 12 
# # . # # . 
# . . # # . 
. . . . # . 
. # # . . . 

[Rank] of [total]: No6 of 12 
. . # # . # 
. . # . # . 
# . . . . . 
. . . # # # 

[Rank] of [total]: No7 of 12 
# # . # # . 
. # # . . . 
. . . . . # 
. . . # # . 

[Rank] of [total]: No8 of 12 
. # . . . . 
# . # . # . 
. . . # . # 
# . # # # . 

[Rank] of [total]: No9 of 12 
. . . . . # 
. . # . . . 
. . # . . # 
. . # # . . 

[Rank] of [total]: No10 of 12 
. . . . # . 
# . . . . . 
. . # # . . 
. . . # . # 

[Rank] of [total]: No11 of 12 
. # . . # . 
. # . # # . 
. . . # . . 
. # . # . # 

---Global Array--- 
. . # . . # . . # # . . . # # # . # . . # # # # 
� � < * � � e { � � � � �  � 
    J      









*** Error in `./gather2darray': double free or corruption (out): 0x0000000001e4c050 *** 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 
*** stack smashing detected ***: ./gather2darray terminated 

=================================================================================== 
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES 
= PID 10979 RUNNING AT User 
= EXIT CODE: 139 
= CLEANING UP REMAINING PROCESSES 
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES 
=================================================================================== 
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) 
This typically refers to a problem with your application. 
Please see the FAQ page for debugging suggestions 

幫助將不勝感激。提前致謝。

Answer 1

你的代碼幾乎是正確的，你只是忘記了一個MPI的重要原則。在MPI函數上使用數組時，MPI會假定您的數組內存是連續分配的。所以你必須改變你的2個dims數組的分配。

#include <stdio.h> 
    #include <stdlib.h> 
    #include <mpi.h> 
    #include <time.h> 

    void print2dCharArray(char** array, int rows, int columns); 


    int main(int argc, char** argv) 
    { 
    int master = 0, np, rank; 
    char version[10]; 
    char processorName[20]; 
    int strLen[10]; 

    // Initialize MPI environment 
    MPI_Init(&argc, &argv); 

    MPI_Comm_size(MPI_COMM_WORLD, &np); 
    if (np != 12) { MPI_Abort(MPI_COMM_WORLD,1); } 
    MPI_Comm_rank(MPI_COMM_WORLD, &rank); 

    // We need a different seed for each process 
    srand(time(0)^(rank * 33/4)); 

    int nDims = 2;    // array dimensions 
    int rows = 4, columns = 6; // rows and columns of each block 
    int prows = 3, pcolumns = 4; // rows and columns of blocks. Each block is handled by 1 process 

    char* pre_grid = (char*) malloc(rows * columns * sizeof(char)); 
    char** grid = (char**) malloc(rows * sizeof(char*)); 
    for (int i = 0; i < rows; i++) 
     grid[i] = &(pre_grid[i * columns]); 

    char** universe = NULL;   // Global array 
    char* pre_universe = NULL; 
    char* recvPtr;     // Pointer to start of Global array 
    int Rows = rows * prows;   // Global array rows 
    int Columns = columns * pcolumns; // Global array columns 
    int sizes[2];      // No of elements in each dimension of the whole array 
    int subSizes[2];     // No of elements in each dimension of the subarray 
    int startCoords[2];    // Starting coordinates of each subarray 
    MPI_Datatype recvBlock, recvMagicBlock; 

    if (rank == master){   // For the master's eyes only 
    /* universe = malloc(Rows * sizeof(char*));*/ 
    /* for (int i = 0; i < Rows; i++)*/ 
    /*  universe[i] = malloc(Columns * sizeof(char));*/ 

     pre_universe = (char*) malloc(Rows * Columns * sizeof(char)); 
     universe = (char**) malloc(Rows * sizeof(char*)); 
     for (int i = 0; i < Rows; i++) { 
      universe[i] = &(pre_universe[i * Columns]); 
     } 



     // Create a subarray (a rectangular block) datatype from a regular, 2d array 
     sizes[0] = Rows; 
     sizes[1] = Columns; 
     subSizes[0] = rows; 
     subSizes[1] = columns; 
     startCoords[0] = 0; 
     startCoords[1] = 0; 

     MPI_Type_create_subarray(nDims, sizes, subSizes, startCoords, MPI_ORDER_C, MPI_CHAR, &recvBlock); 

     // Now modify the newly created datatype to fit our needs, by specifying 
     // (lower bound remains the same = 0) 
     // - new extent 
     // The new region/block will now "change" sooner, as soon as we reach a region of elements 
     //   occupied by a new block, ie. every: (columns) * sizeof(elementType) = 
     MPI_Type_create_resized(recvBlock, 0, columns * sizeof(char), &recvMagicBlock); 

     MPI_Type_commit(&recvMagicBlock); 
     recvPtr = &universe[0][0]; 
    } 

    // populate arrays 
    for (int y = 0; y < rows; y++){ 
     for (int x = 0; x < columns; x++){ 
     grid[y][x] = rank + 65; 
     } 
    } 


    // display local array 
    for (int i = 0; i < np; i++){ 
     if (i == rank) { 
     printf("\n[Rank] of [total]: No%d of %d\n", rank, np); 
     print2dCharArray(grid, rows, columns); 
     } 
     MPI_Barrier(MPI_COMM_WORLD); 
    } 


    /* MPI_Gathering.. */ 
    int recvCounts[np], displacements[np]; 

    // recvCounts: how many chunks of data each process has -- in units of blocks here -- 
    for (int i = 0; i < np; i++) 
     recvCounts[i] = 1; 

    // prows * pcolumns = np 
    // displacements: displacement relative to global buffer (universe) at which to place the 
    //        incoming data block from process i -- in block extents! -- 
    int index = 0; 
    for (int p_row = 0; p_row < prows; p_row++) 
     for (int p_column = 0; p_column < pcolumns; p_column++) 
     displacements[index++] = p_column + p_row * (rows * pcolumns); 

    // MPI_Gatherv(...) is a collective routine 
    // Gather the local arrays to the global array in the master process 
    // send type: MPI_CHAR  (a char) 
    // recv type: recvMagicBlock (a block) 
    MPI_Gatherv(&grid[0][0], rows * columns, MPI_CHAR, //: parameters relevant to sender 
      recvPtr, recvCounts, displacements, recvMagicBlock, master, //: parameters relevant to receiver 
      MPI_COMM_WORLD); 

    // display global array 
    MPI_Barrier(MPI_COMM_WORLD); 
    if (rank == master){ 
     printf("\n---Global Array---\n"); 
     print2dCharArray(universe, Rows, Columns); 
    } 

    free(grid[0]); 
    free(grid); 
    if (rank == master) { 
     free(universe[0]); 
     free(universe); 
     MPI_Type_free(&recvMagicBlock); 
     MPI_Type_free(&recvBlock); 
    } 


    MPI_Finalize(); 
    return 0; 
    } 


    void print2dCharArray(char** array, int rows, int columns) 
    { 
    int i, j; 
    for (i = 0; i < rows; i++){ 
     for (j = 0; j < columns; j++){ 
     printf("%c ", array[i][j]); 
     } 
     printf("\n"); 
    } 
    fflush(stdout); 
    } 

輸出：

---Global Array--- 
    A A A A A A B B B B B B C C C C C C D D D D D D 
    A A A A A A B B B B B B C C C C C C D D D D D D 
    A A A A A A B B B B B B C C C C C C D D D D D D 
    A A A A A A B B B B B B C C C C C C D D D D D D 
    E E E E E E F F F F F F G G G G G G H H H H H H 
    E E E E E E F F F F F F G G G G G G H H H H H H 
    E E E E E E F F F F F F G G G G G G H H H H H H 
    E E E E E E F F F F F F G G G G G G H H H H H H 
    I I I I I I J J J J J J K K K K K K L L L L L L 
    I I I I I I J J J J J J K K K K K K L L L L L L 
    I I I I I I J J J J J J K K K K K K L L L L L L 
    I I I I I I J J J J J J K K K K K K L L L L L L