Thanks for pointers to the changes. Two comments:
1. Example 4.9 has a lot in common with Global Arrays implementation on the SP
or Paragon. However, the implementation of accumulate seems unnecesarily
complicated and might lead to some problems. I am referring to processing of
accumulate request in two, instead of just one, handler functions. In fact, the
same effect could be accomplished by performing accumulate directly from
window_handler, surrounding the sensitive part of code by MPI_handler_lock and
MPI_handler_unlock to assure the atomicity. On some platforms, processing an
additional interrupt adds a substantial overhead. Also, allocating and not
freeing memory buffer in window_handler for the future use in acc_handler is not
that safe. There might be many following invocations of window_handler
(accumulate called by other processors), each attempting to allocate a
substantial amount of memory, before the acc_handler (that frees the memory) is
called. This problem would not exist if entire accumulate request was processed
in window_handler function. Other than that the example seems to be quite
informative and useful.
2. Users might need more control over completion status of remote data
transfers than MPI_FENCE currently allows. We could add a new function,
say MPI_FENCE_INIT, that limits the range of remote transfers that are
traced for completion.
For example:
MPI_PUT(..., rank, ..., comm)
MPI_FENCE_INIT(comm, rank)
MPI_PUT(..., rank, ..., comm)
MPI_FENCE(comm,rank)
which would block the calling process until 2nd put completes while the first one
still might be in progress. Calling MPI_FENCE without the matching MPI_FENCE_INIT
call would be an error. MPI_FENCE_INIT and MPI_FENCE would surround the remote
transfer operations that the user wants to be completed rather than enforcing
completion of all of them.
Jarek