In principle, I like this suggestion a lot. I don't like:
(left group) (right group)
if (rank == root) MPI_Bcast(...)
MPI_Bcast(...);
MPI_Bcast(...); if (rank == root)
MPI_Bcast(...);
which is what you have to do under our proposal to trade
messages. I much prefer:
MPI_Bcast(...) MPI_Bcast(...)
However, when I try to write the argument list for
bcast, it doesn't look quite as pretty (although I still like
the idea):
MPI_Bcast(sendbuf, sendcnt, sendtype,
recvbuf, recvcnt, recvtype, localroot, remoteroot, comm);
You still have to specify two roots since the root must know that he
is the root on the receiving side and you must specify the remote root.
In general this is a problem with all rooted operations.
> Non-rooted operations (alltoall, allgather, allreduce).
> In an alltoall communication, each process in group A is sending data to each
> process in group B, and vice versa. Similarly, for the other operations.
For alltoall, allgather, allgatherv this notion works out beautifully.
For allreduce and reduce_scatter, we would need additional arguments
if it was to be completely general. For example:
allreduce ( sendbuf, recvbuf, count, datatype, op, comm )
will need to be:
allreduce ( sendbuf, sendcount, sendtype,
recvbuf, recvcount, recvtype, op, comm )
> Advantages:
>
> No need for new syntax. We can reserve the use of special buffer arguments
> for in-place collective operations, following the proposal of Fleishman.
I think you do need new argument lists for the rooted calls and some of
the non-rooted calls (whereas under the MSU proposal you do not).
> It is consistent with the definition of collective operations on
> intracommunicators. Think of an intracommunicator as an
> intercommunicator where the two groups happen to coincide. This
> viewpoint is consistent with the way point-to-point communication
> works, and with the definition of the various communicator accessor
> functions. Then intracommunicator collective communication calls
> are a particular case of the intercommunicator calls, with the
> additional coercion that a process that belongs to both groups must
> provide the same parameter list for its "roles" in each group (this
> is necessary, since the process executes only one call. In fact, it
> is quite likely that the pseudocode that we outline in the standard,
> that explains collective communication in terms of point-to-point
> can be used with no modifications for collective communication on
> intracommunicator.
I think that this is definitely the way we should think about
intercommuncator collectives. I like the symmetry of Marc's proposal
but I don't like the way MPI_Bcast looks.
-- Nathan Doss doss@ERC.MsState.Edu