- - - - - - -
We have been reinvestigating the issue of channels for MPI-2. We
evidently left them out in MPI-1 (as noted below), and I would like to
be given the two sides of the rationale, which I believe Mssrs. Snir
and Cownie and others will remember, whereas I do not, unfortunately.
So, I have tried to reconstruct rationale...
MPI_SENDRECV() and MPI_SENDRECV_REPLACE() have no persistent analogues.
This is probably because they can be emulated with two persistent
operations. Also, as the standard says, SENDRECV's can be matched
with RECV's and SEND's, as appropriate. Hence, although a SENDRECV
where the processes pair off in an exchange appears to be a channel,
it is not very optimizable. SENDRECV_INIT would not be very optimizable
either, because there is no guarantee that persistent handles are used
elsewhere.
A SENDRECV that is a channel-like exchange (and unidirectional as a
special case using a count=0), and has a persistent analogue, does
appear to have significant opportunities for performance enhancement.
The relaxed matching up with plain receives would have to be
sacrificed, but more performance could be obtained:
* Negotiation of advance reservations on network or PVCs ...
* Addressing any special issues of alignment/misalignment
* Negotiation for who does any needed data conversion
* Change of protocol as function of length of data
For instance:
MPI_CHANNEL_INIT(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf,
recvcount, recvtype, source, recvtag, comm, request)
[IN sendbuf] initial address of send buffer (choice)
[IN sendcount] number of elements in send buffer (integer)
...
[OUT request] request object (persistent request)
Which is analogous to MPI_SENDRECV. MPI_START, MPI_STARTALL, and
MPI_WAIT/TEST family to be used as with other non-blocking operations
to convert request -> status, and mark as complete the operation. The
CHANNEL operations would operate logically out of band from normal
send/recv, and using the same tags as are used in SEND/RECV-type calls
would be erroneous. The MPI_CHANNEL_INIT would reserve the tags
mentioned in the call. Use of wildcard tags/srcs in the CHANNEL call
would be erroneous. Tags are freed up for other uses when the
MPI_REQUEST_FREE() occurs.
For completeness, the syntactic sugar of an MPI_CHANNEL(), equivalent
to MPI_CHANNEL_INIT()+MPI_START()+MPI_WAIT()+MPI_REQUEST_FREE() would
be provided as well.
MPI_CHANNEL(sendbuf, sendcount, sendtype, dest, sendtag, recvbuf,
recvcount, recvtype, source, recvtag, comm, status)
[IN sendbuf] initial address of send buffer (choice)
[IN sendcount] number of elements in send buffer (integer)
...
[OUT status] status object (Status)
- - - - - -
For the realtime discussion, further versions of this channel that provide
QoS properties appear very interesting, but I will restrict my remarks
to the realtime subcommittee reflector on this front.
-Tony Skjellum