To: SNIR
cc: Mpi-1Sided @ mcs.anl.gov (bcc: Marc Snir/Watson/IBM Research)
Subject: Re: Reasons for negative votes on 1-sided at Sept. MPI mtg
--0__=A6YrsraojRKU2qQZAhHuCy0Y6C0Vg4K4r2aEs5fiysbNA9IeaskGnINn
******
Re: barrier
The current text needs to be changed -- RMA_Barrier acts as a barrier,
and the weak/strong option may not be useful. That said, it is still
true that the implementation may decide not to push data as soon as a put
call occur, but push out data only when the ensuing call to rma_barrier
occurs. MPI_PUT is a nonblocking call, and returns before data has been
copied out of memory. It only has to complete at the following
synchronization call, that acts as a wait call for all RMA calls issued
since the preceding synchronization.
********
I will try to make it clear. When using POST/WAIT/START/COMPLETE, the
target can specify a period during which other processes can access its
window -- i.e. the period between the START and COMPLETE. This is the
same in the P/G/O proposal -- i.e. the period during which OFFER is
executing. If RMA operations are executed during this time in other
processes, they can access the window immediately (if the implementation
wants to). If RMA operations are executed at other times, they may block
or they could be delayed internally, or (if they are PUTs) they might
buffer their data, perhaps on the target processor. It is up to the MPI
implementor to do something good for that architecture.
However, when using RMA_BARRIER, there is no way for each processor to
specify the period during which other processes can access its window.
So, if I am on process X, and I am reading a buffer (i.e.
window) which another process Y wants to PUT data into, how do I keep
Y from messing up the data I am reading? I believe that I (process X)
must keep it (process Y) from executing the PUT by using some
synchronization (another barrier or a message or something). This is the
case I am objecting to. The PUT cannot even be allowed to be called on
the origin until the target is ready for it, because there is no other
way to postpone it. The P/G/O proposal, on the other hand, works exactly
the same way for both the collective and non-collective cases (i.e.
OFFERC and OFFER). This allows the possibility of much more efficient
implementations of collective, especially on distributed memory machines
where much data is being exchanged, because in the collective case,
processes only need to block about half as often. That is, they can omit
the barrier that holds off PUTs from occurring too soon, and just leave
the barrier (or in my case, OFFERC) that waits for the PUTs to finish.
******
Withe RMA_BARRIER, the synchronization patter is going to be
everybody puts
barrier
everybody does local updates
barrier
....
Or, perhaps,
odd gets, even does local updates
barrier
even gets, odd does local accesses
barrier
Or, the windows are split into two windows, and program alternates from one
set of window to the other, and
so on.
I agree that RMA_BARRIER works differently than post/wait/start/complete,
because the same call is used
both to bracket put/get calls and to bracket local accesses to a window. The
alternative that you suggest
is to have a collective call to expose your window to remote accesses, and a
collective
call to access remote windows. I am happy with the current design
because I believe it is closest to the programming style people use with
put/get calls
today, and I believe it fits better MPI paradigm of a (symmetric) collective
call.
When people want more control on communication, they go to
post/wait/start/complete.
******
--0__=A6YrsraojRKU2qQZAhHuCy0Y6C0Vg4K4r2aEs5fiysbNA9IeaskGnINn--