> David DiNucci writes:
> -] ... If dynamic allocation and communication are combined (as they
> -] are in CDS1 and the Reactive Kernel and Leslie Hart's proposal), then any
> -] desired alignment for simple transfer can be had, essentially for free.
>
> This is true for dynamic memory allocation but what prevents me
> from declaring a variable such as
>
> char MyBuffer[1024];
>
> and sending a message such as
>
> MPI_Send( &MyBuffer[1], 1, MPI_CHAR, 1,1, MPI_COMM_WORLD );
>
> This isn't a problem so now what if we wish to do something
> similar with a put/get (using memcpy() like parameters)
>
> MPI_Put( &DestBuff[1], &MyBuffer[1], sizeof(char) * 1); ?
>
> Now we have a possible alignment problem.
Yes. An unaligned "put" command like that isn't possible in any of the three
"RK-ish" proposals I mentioned. But maybe your comment is "It's not a problem
with message passing, so you should be able to do it using one-sided, too". It
is certainly just as hard to place an unaligned message in user space as an
unaligned one-sided communication. Maybe unaligned messages should be phased
out as well.
As crazy as this may sound, that is the approach taken in the original Reactive
Kernel and in CDS1 (though, perhaps, not entirely with this motivation). All
communication is performed using dynamic-memory-style aligned buffer
communication, and if you want the data in some particular place, you copy
(and translate, if necessary) to/from the comm buffer. CDS1 also has built
routines (called "send" and "recv") that combine the communication and the
copy/translation, so they end up looking almost like similar routines in MPI.
If the user process needs both the communication and the copy, then he/she
can call these, opening the possibility of optimizing the communication+copy
on some architectures. In any case, the user can always be guaranteed pretty
good performance on any architecture if all they need is communication.
Needless to say, retrofitting MPI to use this approach would probably not be
possible, since the semantics of point-to-point are in stone.
> Thinking about how to implement it on an architecture that only
> allows put/get -s on X-bit boundaries gets tough. For data not
> aligned on X-bit boundaries, 1-sided suddenly becomes 2-sided
> which sort of steps all over the whole idea behind there being
> 1-sided communications.
It depends on what that idea is. See below.
>
> This brings up the point of "Well, let's just limit it to the worst
> case alignment among the platforms." This would imply 64-bit
> alignment but wait, there are machines that can't do shared
> memory put/get -s at all so the worst case is no 1-sided at all.
> This approach would mean that 1-sided should actually not be
> in the standard.
But by that reasoning, it sounds like 2-sided shouldn't be in the standard,
since it often requires unaligned transfers.
>
> .. Of course, this is no longer a 1-sided
> communication and in fact, it will be kinda slow - defeating all the
> purposes/advantages of 1-sided communications in one fell swoop.
If a remote machine must perform *any* specific command before accessing the
results of a "put" (and it may be required to perform such a command unless
there is a very simple and fully consistent memory structure), how is that
"put" semantically different from a send, and the specific command different
from a receive? Apparently, the difference is the ability for the "putter" to
specify where the data should appear in the remote machine (or, similarly,
for the "getter" to specify where it obtains the data from the remote machine).
Of course, you want some kinds of control over legal "wheres". I don't think
1-sided really has anything to do with the hardware utilized, or the speed of
the operation.
In MPI, the "where" is specified as a memory address, which (in its current
incarnation) might be unaligned. In CDS1, the "where" is specified as a cell
id, which is effectively always aligned. (A cell id can be regarded as
analogous to a message tag, which can, in some respects, be considered as an
abstract address.)
>
> Shane
>
I hope that some of my perspectives are more helpful than confusing.
-Dave
===============================================================================
David C. DiNucci | MRJ, Inc., Rsrch Scntst |USMail: NASA Ames Rsrch Ctr
dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2
(415)604-4430 | Parallel Tools Group | Moffett Field, CA 94035