At Sandia National Labs, we are developing the Puma operating
system (successor the the Sunmos operating system). Puma
uses message passing primitives known as "portals" to support
interprocess communication within a distributed memory machine.
Portals are fundamentally designed to send and receive messages
directly from/to a process' address space, thus avoiding costly
memory copies.
Two of the portal primitives include read-memory and write-memory to
a "window" in a process' address space on a remote node. There is
no need to map in the address space of the remote process. The
local process simply needs to know that a window exists on the
remote node and that it has permission to access it. The remote
node controls access to the window. Note that this model is
very similar to the current get/put proposals. (I'm afraid
I don't fully understand Eric's use of "arbitrary" with respect
to remote process address'. This may be a shared memory problem
rather than a distributed memory problem. In MPP systems,
the remote process can be very specific about the target address,
and the local process doesn't care.)
It seems to me that the shared buffer description provided
below can be implemented under the current specification. First,
a shared memory region can be allocated, and then it can be
setup as a get/put window. I don't mind supporting a SHMALLOC
routine, but I definitely want the ability to have the remote node
DECIDE what part of memory to use for get/put operations.
I'm not sure what a shared memory buffer would look like on an
MPP machine, but I would be opposed to any buffer setup that
would require memory copies on either the local or remote side.
A purely shmalloc model tells me that I may need to memcpy
under many circumstances.
In this message, I just wanted to let you know that there is
an up and coming distributed memory OS that can efficiently
implement the current get/put proposals (hrecv is not an
implementation option for us). I am interested in simplifying
current proposals if that is possible, but otherwise feel
that these proposals are on track.
Lance
===========================================================================
Lance Shuler W: (505) 844-2159
Parallel Computing Sciences Department, 1424 Fax: (505) 845-7442
Sandia National Labs
Albuquerque, NM 87185-1109 shuler@cs.sandia.gov
===========================================================================