I propose an alternative design where collective operations on
intercommunicators be symmetic: each group is both sending and receiving;
data sent by one group is received by the other.
Thus:
Rooted operations (bcast, scatter, gather, reduce).
Each group provides a root, that communicates with all processes in the other
group. Thus in a broadcast, the data sent by the root in group A is received
by all processes in group B, and vice versa.
Non-rooted operations (alltoall, allgather, allreduce).
In an alltoall communication, each process in group A is sending data to each
process in group B, and vice versa. Similarly, for the other operations.
Advantages:
No need for new syntax. We can reserve the use of special buffer arguments
for in-place collective operations, following the proposal of Fleishman.
It is useful. For example, in a transpose of a block-distributed matrix, one
will have either a regular alltoall communication, or a bipartite alltoall
communication, where all processes in one group send data to the processes
in the other group, and vice-versa.
It generalizes the MSU proposal: one can always get a unidirectional
collective communication on an intercommunicator by sending zero data in the
other direction.
It is consistent with the definition of collective operations on
intracommunicators. Think of an intracommunicator as an intercommunicator
where the two groups happen to coincide. This viewpoint is consistent with
the way point-to-point communication works, and with the definition of the
various communicator accessor functions. Then intracommunicator collective
communication calls are a particular case of the intercommunicator calls,
with the additional coercion that a process that belongs to both groups must
provide the same parameter list for its "roles" in each group (this is
necessary, since the process executes only one call. In fact, it is quite
likely that the pseudocode that we outline in the standard, that explains colle
ctive communication in terms of point-to-point can be used with no
modifications for collective communication on intracommunicator.