127. Communicator Constructors

Up: Communicator Management Next: Communicator Destructors Previous: Communicator Accessors

The following are collective functions that are invoked by all processes in the group or groups associated with comm.

Note that there is a chicken-and-egg aspect to MPI in that a communicator is needed to create a new communicator. The base communicator for all MPI communicators is predefined outside of MPI, and is MPI_COMM_WORLD. This model was arrived at after considerable debate, and was chosen to increase ``safety'' of programs written in MPI. ( End of rationale.)

The MPI interface provides four communicator construction routines that apply to both intracommunicators and intercommunicators. The construction routine MPI_INTERCOMM_CREATE (discussed later) applies only to intercommunicators.

An intracommunicator involves a single group while an intercommunicator involves two groups. Where the following discussions address intercommunicator semantics, the two groups in an intercommunicator are called the left and right groups. A process in an intercommunicator is a member of either the left or the right group. From the point of view of that process, the group that the process is a member of is called the local group; the other group (relative to that process) is the remote group. The left and right group labels give us a way to describe the two groups in an intercommunicator that is not relative to any particular process (as the local and remote groups are).
MPI_COMM_DUP(comm, newcomm)
IN comm communicator (handle)
OUT newcomm copy of comm (handle)

int MPI_Comm_dup(MPI_Comm comm, MPI_Comm *newcomm)

MPI::Intracomm MPI::Intracomm::Dup() const
MPI::Intercomm MPI::Intercomm::Dup() const
MPI::Cartcomm MPI::Cartcomm::Dup() const
MPI::Graphcomm MPI::Graphcomm::Dup() const
MPI::Comm& MPI::Comm::Clone() const = 0
MPI::Intracomm& MPI::Intracomm::Clone() const
MPI::Intercomm& MPI::Intercomm::Clone() const
MPI::Cartcomm& MPI::Cartcomm::Clone() const
MPI::Graphcomm& MPI::Graphcomm::Clone() const
MPI_COMM_DUP Duplicates the existing communicator comm with associated key values. For each key value, the respective copy callback function determines the attribute value associated with this key in the new communicator; one particular action that a copy callback may take is to delete the attribute from the new communicator. Returns in newcomm a new communicator with the same group or groups, any copied cached information, but a new context (see Section Functionality ). Please see Section Communicators on page Communicators for further discussion about the C++ bindings for Dup() and Clone().
Advice to users.

This operation is used to provide a parallel library call with a duplicate communication space that has the same properties as the original communicator. This includes any attributes (see below), and topologies (see Chapter Process Topologies ). This call is valid even if there are pending point-to-point communications involving the communicator comm. A typical call might involve a MPI_COMM_DUP at the beginning of the parallel call, and an MPI_COMM_FREE of that duplicated communicator at the end of the call. Other models of communicator management are also possible.

This call applies to both intra- and inter-communicators. ( End of advice to users.)

Advice to implementors.

One need not actually copy the group information, but only add a new reference and increment the reference count. Copy on write can be used for the cached information. ( End of advice to implementors.)
MPI_COMM_CREATE(comm, group, newcomm)
IN commcommunicator (handle)
IN group Group, which is a subset of the group of comm (handle)
OUT newcomm new communicator (handle)

int MPI_Comm_create(MPI_Comm comm, MPI_Group group, MPI_Comm *newcomm)

MPI::Intercomm MPI::Intercomm::Create(const MPI::Group& group) const
MPI::Intracomm MPI::Intracomm::Create(const MPI::Group& group) const
If comm is an intra-communicator, this function creates a new communicator newcomm with communication group defined by group and a new context. No cached information propagates from comm to newcomm. The function returns MPI_COMM_NULL to processes that are not in group. The call is erroneous if not all group arguments have the same value, or if group is not a subset of the group associated with comm. Note that the call is to be executed by all processes in comm, even if they do not belong to the new group.

The requirement that the entire group of comm participate in the call stems from the following considerations:

( End of rationale.)

Advice to users.

MPI_COMM_CREATE provides a means to subset a group of processes for the purpose of separate MIMD computation, with separate communication space. newcomm, which emerges from MPI_COMM_CREATE can be used in subsequent calls to MPI_COMM_CREATE (or other communicator constructors) further to subdivide a computation into parallel sub-computations. A more general service is provided by MPI_COMM_SPLIT, below. ( End of advice to users.)

Advice to implementors.

Since all processes calling MPI_COMM_DUP or
MPI_COMM_CREATE provide the same group argument, it is theoretically possible to agree on a group-wide unique context with no communication. However, local execution of these functions requires use of a larger context name space and reduces error checking. Implementations may strike various compromises between these conflicting goals, such as bulk allocation of multiple contexts in one collective operation.

Important: If new communicators are created without synchronizing the processes involved then the communication system should be able to cope with messages arriving in a context that has not yet been allocated at the receiving process. ( End of advice to implementors.)
If comm is an intercommunicator, then the output communicator is also an intercommunicator where the local group consists only of those processes contained in group (see Figure 13 ). The group argument should only contain those processes in the local group of the input intercommunicator that are to be a part of newcomm. If either group does not specify at least one process in the local group of the intercommunicator, or if the calling process is not included in the group, MPI_COMM_NULL is returned.


In the case where either the left or right group is empty, a null communicator is returned instead of an intercommunicator with MPI_GROUP_EMPTY because the side with the empty group must return MPI_COMM_NULL. ( End of rationale.)

Figure 13: Intercommunicator create using MPI_COMM_CREATE extended to intercommunicators. The input groups are those in the grey circle.

Example The following example illustrates how the first node in the left side of an intercommunicator could be joined with all members on the right side of an intercommunicator to form a new intercommunicator.

        MPI_Comm  inter_comm, new_inter_comm; 
        MPI_Group local_group, group; 
        int       rank = 0; /* rank on left side to include in  
                               new inter-comm */ 
        /* Construct the original intercommunicator: "inter_comm" */ 
        /* Construct the group of processes to be in new  
           intercommunicator */ 
        if (/* I'm on the left side of the intercommunicator */) { 
          MPI_Comm_group ( inter_comm, &local_group ); 
          MPI_Group_incl ( local_group, 1, &rank, &group ); 
          MPI_Group_free ( &local_group ); 
          MPI_Comm_group ( inter_comm, &group ); 
        MPI_Comm_create ( inter_comm, group, &new_inter_comm ); 
        MPI_Group_free( &group ); 
MPI_COMM_SPLIT(comm, color, key, newcomm)
IN commcommunicator (handle)
IN colorcontrol of subset assignment (integer)
IN key control of rank assigment (integer)
OUT newcomm new communicator (handle)

int MPI_Comm_split(MPI_Comm comm, int color, int key, MPI_Comm *newcomm)

MPI::Intercomm MPI::Intercomm::Split(int color, int key) const
MPI::Intracomm MPI::Intracomm::Split(int color, int key) const
This function partitions the group associated with comm into disjoint subgroups, one for each value of color. Each subgroup contains all processes of the same color. Within each subgroup, the processes are ranked in the order defined by the value of the argument key, with ties broken according to their rank in the old group. A new communicator is created for each subgroup and returned in newcomm. A process may supply the color value MPI_UNDEFINED, in which case newcomm returns MPI_COMM_NULL. This is a collective call, but each process is permitted to provide different values for color and key.

A call to MPI_COMM_CREATE(comm, group, newcomm) is equivalent to
a call to MPI_COMM_SPLIT(comm, color, key, newcomm), where all members of group provide color~ =~0 and key~=~ rank in group, and all processes that are not members of group provide color~ =~ MPI_UNDEFINED. The function MPI_COMM_SPLIT allows more general partitioning of a group into one or more subgroups with optional reordering. The value of color must be nonnegative.

Advice to users.

This is an extremely powerful mechanism for dividing a single communicating group of processes into k subgroups, with k chosen implicitly by the user (by the number of colors asserted over all the processes). Each resulting communicator will be non-overlapping. Such a division could be useful for defining a hierarchy of computations, such as for multigrid, or linear algebra.

Multiple calls to MPI_COMM_SPLIT can be used to overcome the requirement that any call have no overlap of the resulting communicators (each process is of only one color per call). In this way, multiple overlapping communication structures can be created. Creative use of the color and key in such splitting operations is encouraged.

Note that, for a fixed color, the keys need not be unique. It is MPI_COMM_SPLIT's responsibility to sort processes in ascending order according to this key, and to break ties in a consistent way. If all the keys are specified in the same way, then all the processes in a given color will have the relative rank order as they did in their parent group. Essentially, making the key value zero for all processes of a given color means that one doesn't really care about the rank-order of the processes in the new communicator. ( End of advice to users.)


color is restricted to be nonnegative, so as not to confict with the value assigned to MPI_UNDEFINED. ( End of rationale.)
The result of MPI_COMM_SPLIT on an intercommunicator is that those processes on the left with the same color as those processes on the right combine to create a new intercommunicator. The key argument describes the relative rank of processes on each side of the intercommunicator (see Figure 14 ). For those colors that are specified only on one side of the intercommunicator, MPI_COMM_NULL is returned. MPI_COMM_NULL is also returned to those processes that specify MPI_UNDEFINED as the color.

Figure 14: Intercommunicator construction achieved by splitting an existing intercommunicator with MPI_COMM_SPLIT extended to intercommunicators.

Example(Parallel client-server model). The following client code illustrates how clients on the left side of an intercommunicator could be assigned to a single server from a pool of servers on the right side of an intercommunicator.

        /* Client code */ 
        MPI_Comm  multiple_server_comm; 
        MPI_Comm  single_server_comm; 
        int       color, rank, num_servers; 
        /* Create intercommunicator with clients and servers:  
           multiple_server_comm */ 
        /* Find out the number of servers available */ 
        MPI_Comm_remote_size ( multiple_server_comm, &num_servers ); 
        /* Determine my color */ 
        MPI_Comm_rank ( multiple_server_comm, &rank ); 
        color = rank % num_servers; 
        /* Split the intercommunicator */ 
        MPI_Comm_split ( multiple_server_comm, color, rank,  
                         &single_server_comm ); 
The following is the corresponding server code:
        /* Server code */ 
        MPI_Comm  multiple_client_comm; 
        MPI_Comm  single_server_comm; 
        int       rank; 
        /* Create intercommunicator with clients and servers:  
           multiple_client_comm */ 
        /* Split the intercommunicator for a single server per group 
           of clients */ 
        MPI_Comm_rank ( multiple_client_comm, &rank ); 
        MPI_Comm_split ( multiple_client_comm, rank, 0,  
                         &single_server_comm );   

Up: Communicator Management Next: Communicator Destructors Previous: Communicator Accessors

Return to MPI-2.1 Standard Index
Return to MPI Forum Home Page

MPI-2.0 of July 1, 2008
HTML Generated on July 6, 2008