mpi從一個通信器到另一個通信器的集體操作

Question

我有一個與MPI並行的應用程序，它被分成許多不同的任務。每個處理器只分配一個任務，並且分配有相同任務的處理器組被分配給它自己的通信器。定期地，任務需要同步。目前，通過MPI_COMM_WORLD完成同步，但缺點是不能使用集體操作，因爲無法保證其他任務能夠達到該代碼塊。

作爲更具體的例子：

task1: equation1_solver, N nodes, communicator: mpi_comm_solver1 
task2: equation2_solver, M nodes, communicator: mpi_comm_solver2 
task3: file IO   , 1 node , communicator: mpi_comm_io 

我想MPI_SUM上任務1的陣列，並具有結果顯示在TASK3。有沒有一種有效的方法來做到這一點？ （我很抱歉，如果這是一個愚蠢的問題，我沒有太多的經驗，創建和使用自定義MPI通信器）

Answer 1

Charles是完全正確的;互相溝通者允許你在溝通者之間進行交談（或者在這種情況下區分「正常」溝通者，「溝通者內部溝通者」，這對我沒有太大的改進）。

我一直髮現使用這些互聯通訊器對於那些新手來說有點令人困惑。不是基本的想法，這是有道理的，但使用（說）MPI_Reduce與其中之一的機制。執行縮減的任務組指定遠程通信器上的根級別，迄今爲止非常好;但在遠程排名溝通者，大家不是根指定MPI_PROC_NULL作爲根，而實際的根指定MPI_ROOT。人們爲了向後兼容而做的事情，嘿？

#include <mpi.h> 
#include <stdio.h> 


int main(int argc, char **argv) 
{ 
    int commnum = 0;   /* which of the 3 comms I belong to */ 
    MPI_Comm mycomm;  /* Communicator I belong to */ 
    MPI_Comm intercomm; /* inter-communicator */ 
    int cw_rank, cw_size; /* size, rank in MPI_COMM_WORLD */ 
    int rank;    /* rank in local communicator */ 

    MPI_Init(&argc, &argv); 
    MPI_Comm_rank(MPI_COMM_WORLD, &cw_rank); 
    MPI_Comm_size(MPI_COMM_WORLD, &cw_size); 

    if (cw_rank == cw_size-1)  /* last task is IO task */ 
     commnum = 2; 
    else { 
     if (cw_rank < (cw_size-1)/2) 
      commnum = 0; 
     else 
      commnum = 1; 
    } 

    printf("Rank %d in comm %d\n", cw_rank, commnum); 

    /* create the local communicator, mycomm */ 
    MPI_Comm_split(MPI_COMM_WORLD, commnum, cw_rank, &mycomm); 

    const int lldr_tag = 1; 
    const int intercomm_tag = 2; 
    if (commnum == 0) { 
     /* comm 0 needs to communicate with comm 2. */ 
     /* create an intercommunicator: */ 

     /* rank 0 in our new communicator will be the "local leader" 
     * of this commuicator for the purpose of the intercommuniator */ 
     int local_leader = 0; 

     /* Now, since we're not part of the other communicator (and vice 
     * versa) we have to refer to the "remote leader" in terms of its 
     * rank in COMM_WORLD. For us, that's easy; the remote leader 
     * in the IO comm is defined to be cw_size-1, because that's the 
     * only task in that comm. But for them, it's harder. So we'll 
     * send that task the id of our local leader. */ 

     /* find out which rank in COMM_WORLD is the local leader */ 
     MPI_Comm_rank(mycomm, &rank); 

     if (rank == 0) 
      MPI_Send(&cw_rank, 1, MPI_INT, cw_size-1, 1, MPI_COMM_WORLD); 
     /* now create the inter-communicator */ 
     MPI_Intercomm_create(mycomm, local_leader, 
           MPI_COMM_WORLD, cw_size-1, 
           intercomm_tag, &intercomm); 
    } 
    else if (commnum == 2) 
    { 
     /* there's only one task in this comm */ 
     int local_leader = 0; 
     int rmt_ldr; 
     MPI_Status s; 
     MPI_Recv(&rmt_ldr, 1, MPI_INT, MPI_ANY_SOURCE, lldr_tag, MPI_COMM_WORLD, &s); 
     MPI_Intercomm_create(mycomm, local_leader, 
           MPI_COMM_WORLD, rmt_ldr, 
           intercomm_tag, &intercomm); 
    } 


    /* now let's play with our communicators and make sure they work */ 

    if (commnum == 0) { 
     int max_of_ranks = 0; 
     /* try it internally; */ 
     MPI_Reduce(&rank, &max_of_ranks, 1, MPI_INT, MPI_MAX, 0, mycomm); 
     if (rank == 0) { 
      printf("Within comm 0: maximum of ranks is %d\n", max_of_ranks); 
      printf("Within comm 0: sum of ranks should be %d\n", max_of_ranks*(max_of_ranks+1)/2); 
     } 

     /* now try summing it to the other comm */ 
     /* the "root" parameter here is the root in the remote group */ 
     MPI_Reduce(&rank, &max_of_ranks, 1, MPI_INT, MPI_SUM, 0, intercomm); 
    } 

    if (commnum == 2) { 
     int sum_of_ranks = -999; 
     int rootproc; 

     /* get reduction data from other comm */ 

     if (rank == 0) /* am I the root of this reduce? */ 
      rootproc = MPI_ROOT; 
     else 
      rootproc = MPI_PROC_NULL; 

     MPI_Reduce(&rank, &sum_of_ranks, 1, MPI_INT, MPI_SUM, rootproc, intercomm); 

     if (rank == 0) 
      printf("From comm 2: sum of ranks is %d\n", sum_of_ranks); 
    } 

    if (commnum == 0 || commnum == 2); 
      MPI_Comm_free(&intercomm); 

    MPI_Finalize(); 
} 

Answer 2

所有你需要的是創建一個新的通訊器，包括來自兩個任務，你想溝通的節點。看看MPI小組和傳播者。你可以在網上找到很多例子，here for instance。