This proposal is identical to the one I originally sent out to the 1sided list
back in November, with the exception of a "atomicity" argument that I have
since added to MPI_SHMALLOC(). It replaces all of chapter 4, with the possible
exception of the HRECV material. For those of you who missed the fun the first
time around (or if you would just like to refresh your memory), my full
rationale for this proposal and subsequent discussions are in the archives.
All comments are welcome...
-------------------------------------------------------------------------
MPI_SHMALLOC(buf, len, oldcomm, newcomm, atomicity)
OUT buf location of pointer which will hold the
local address of the window (choice)
IN len length of window in bytes (integer)
IN oldcomm communicator (handle)
OUT newcomm get/put communicator created (handle)
OUT atomicity smallest allowable unit/alignment of data
This collective call allocates a "window" of memory to be used for remote
operations and associates it with a new get/put communicator. The new
communicator has identical membership to the old one. The window is
deallocated by a call to MPI_COMM_FREE().
MPI_GET(buf, count, type, offset, rank, comm)
IN buf address of local destination buffer (choice)
IN count number of elements sent (integer)
IN type datatype of elements sent (handle)
IN offset byte offset within remote window of data to
get (integer)
IN rank rank of remote process (integer)
IN comm get/put communicator (handle)
This call copies data from the get/put window of a remote process into a
local buffer. It returns when the data in buf has been safely sent on its way.
MPI_PUT(buf, count, type, offset, rank, comm)
IN buf address of local source buffer (choice)
IN count number of elements sent (integer)
IN type datatype of elements sent (handle)
IN offset byte offset within remote window of data to
put (integer)
IN rank rank of remote process (integer)
IN comm get/put communicator (handle)
This call copies data from a local buffer into the get/put window of a remote
process. It returns when buf contains the requested data.
Also
----
When called with a get/put communicator, MPI_BARRIER() will block until all
pending remote operations on that communicator have completed.
Discussion
----------
There are obvious non-blocking and/or synchronous extensions to consider for
MPI_GET and MPI_PUT. Let's ignore them all for the moment, they will be
trivial to add later.
The "atomicity" argument is an attempt to deal with common limitations found in
hardware. For example, perhaps a machine can only guarentee atomicity on
longword data when doing puts. There are really two different knobs to turn
here, the size of the smallest atomicity unit and the required alignment that
such a unit must have in memory. In practice, these values are usually the same
and so I've combined them. Are there any machines for which these values are
different?
At the moment, I'm assuming strict ordering between subsequent gets and puts.
That is, a put followed by a get (within the same communicator) will do the
right thing. This seems to be the most useful programming model and I see no
obvious harm to implementors in requiring it.
Interactions between get/put and send/recv are currently undefined; should they
remain so?
When dealing with the window buffers, all addresses are calculated as bytes.
Do we really want to create a new communicator when we allocate the shared
buffers? How many such buffers do we think each process is likely to allocate?
If the number is small, perhaps we could just "attach" the buffers to
communicators that already exist?
It might be a good idea to require processes to pass the same len argument to
MPI_SHMALLOC(). It might also be good to allow NULL for the buf argument,
indicating that a process does not wish to open up a local window but still
wishes access to the windows on other processes.
-- Eric Salo Silicon Graphics Inc. "Do you know what the (415)933-2998 2011 N. Shoreline Blvd, 7L-802 last Xon said, just salo@sgi.com Mountain View, CA 94043-1389 before he died?"