I like this new proposal. As for the order of execution between
the complete_fn and a posted handler routine, I would prefer to state
"The order is unspecified by MPI. However, a good quality
implementation will always use the same order (handler routine first
and complete_fn second, or vice-versa)." (This way, the behavior
for any given implementation is predictable. However, a portable
program should not rely on any order).
"The order is the following: posted handler routine first and complete_fn
second." (Since the handler is associated with a specific request and
the complete_fn with a GR=meta-request, it seems more natural to me
to start with the handler routine first, and then proceed with the
Another possible choice is to say that posting a handler to a request
with a GR, substitutes the complete_fn for that given request (that
instantiation of the GR), and therefore the order issue is irrelevant.
I think we should have a discussion of the pros and cons of each
image moved lederman @ cs.wisc.edu
to file: 12/05/96 12:03 PM
To: mpi-external @ mcs.anl.gov
cc: (bcc: Jean-Pierre Prost/Watson/IBM Research)
Subject: new generalized request
Taking into account the responses from my previous mail and comments
made before SC96, I will now outline my proposal for generalized
requests (GR). I hope it is cleaner and defines the semantics better.
I would like to see what comments people have now so I can hopefully
write it up once. The terminology I use for null, inactive, and
active handle are from the MPI-1 definitions.
GR only have a nonblocking (nonpersistent) form. A template for a GR
is made with MPI_GR_INIT(start_fn, push_fn, complete_fn, cancel_fn,
MPI_Gr *gr_req). This is like the current MPI_META_REQUEST_CREATE but
the extra_state is removed and it returns a request that is a new type
of handle instead of an MPI_Request. Since the gr_req is never used
in MPI_START, MPI_WAIT/TEST, etc. this should make error detection a
little easier. The gr_req is never used directly in an active GR so
it is always "inactive" and you can start it as many times as you want.
A GR is started with
MPI_GR_BEGIN(MPI_Gr gr_req, extra_state, MPI_Request *req).
This starts the GR and returns a standard MPI request. req is active
when MPI_GR_BEGIN returns and causes start_fn to be called. push_fn can go
on later in the "background". Note that extra_state is given here so
you can have unique values for each instantiation of the GR.
MPI_MARK_REQUEST_COMPLETE(MPI_Request req) internally marks the GR as
done. The only type of req allowed in this call is one from an
MPI_GR_BEGIN (i.e., can't do a regular MPI_Request). The req is still
active when this is done.
MPI_WAIT/TEST(req, status) will return true/complete on the GR if
MPI_MARK_REQUEST_COMPLETE has been called. This causes the
complete_fn to be called with the status provided. complete_fn will
finish before wait/test returns. On return, req is deallocated and
the handle is null by setting it to MPI_REQUEST_NULL. By definition
the req is now "inactive".
MPI_GR_FREE(MPI_Gr *gr_req) marks gr_req for deallocation and makes the
handle null by setting it to MPI_GR_NULL. In the MPI way, you can do
this even if there are GR what are outstanding that were created from
As with pt-to-pt you can do a MPI_CANCEL(req) on a req that is a GR.
As with pt-to-pt you still need to do a test/wait on req. Advice to
users is that since complete_fn gets called at the test/wait, you need
to see if req has been cancelled if that matters.
You can mark a GR for deallocation and cause the handle to become null
(MPI_NULL_REQUEST) by calling MPI_REQUEST_FREE(req). The req may
still be active. It will become inactive after
MPI_MARK_REQUEST_COMPLETE finishes. Advice to impl: The complete_fn
can and must be called after MPI_MARK_REQUEST_COMPLETE even if the req
has been freed. The status supplied to complete_fn is MPI_STATUS_NULL
in this case.
Interaction with handlers: If a handler and the complete_fn are both
elegible to execute (for example, the GR finishes while a wait is
active on the GR and a handler is attached to the GR) then MPI makes
no statement about the order in which they will execute. If someone
thinks we need to specify order and has a proposal I am open to this.
Geneal advice: If callbacks in GR (or handlers) change the state of MPI
handles (such as freeing a request), care must be taken that the "main
thread" of code will not generate errors due to this change.
Clarification to MPI-1: To help avoid errors from asynchronous
changes, MPI_CANCEL will accept the null handle MPI_REQUEST_NULL.
When this happens, MPI_CANCEL will simply return without generating an
IBM T.J. Watson Research Center
PO Box 218
Yorktown Heights, NY 10598
Phone: (914) 945 3225 - Fax: (914) 945 2141
Lotus Notes: Jean-Pierre Prost @ IBM Research