I addressed your complaint at the end of my proposal, but I wasn't very clear.
I wrote:
>It could conceivably be more difficult to use these collective operations
>than the existing non-collective operations with BARRIERs in some cases,
>because the collective operations require that all GETs or PUTs be performed
>in one operation, rather than issuing separate GETs and PUTs, followed by
>a BARRIER (which effectively means "now go do them").
I think this is your complaint. Then I wrote:
>If this is a problem, I think it could be remedied by offering other similar
>routines with different argument structures -- i.e. lists of locations rather
>than complex derived types, or even individual calls to collect arguments,
>with a final call that uses the collection (like Xt_Setarg in X).
Let me more fully describe the "or even..." part. Suppose that there was a
routine, called "PUT_ARG", which took exactly the same arguments as the current
PUT (perhaps minus the increment), and another routine called
PUT_GO(recvbuf, comm, rank)
OUT recvbuf
IN comm
IN target_rank
(recvbuf would only be significant in the target process.)
The semantics would be that a PUT_ARG operation would *effectively*
just collect its arguments and store them somewhere (similar to Xt_Setarg
in X), and the PUT_GO would use whatever arguments had been stored up and
perform a collective PUT. That's how the semantics would be described to
the user, at least. Internally, your implementation could modify the
target buffer right away, just as soon it got the PUT_ARG, as long as (a) the
target process had already begun the PUT_GO, and (b) PUT_ARG didn't block
forever waiting for the target process to perform the PUT_ARG.
Now, the processes look like:
Process A Process B
--------- ---------
PUT_ARG PUT_GO
PUT_ARG fetch
PUT_ARG
PUT_GO
This is exactly like your example of the "barrier_only" model, but renaming
PUT to PUT_ARG and BARRIER to PUT_GO. Now consider GET. In the barrier-only
model, it would be: (now A is the target)
Process A Process B
---------- ----------
store
BARRIER BARRIER
GET
GET
GET
BARRIER BARRIER
The second BARRIER is required so that Process A can satisfy the GETs (on
some implementations), and so it can know when it is free to modify the
buffer again. The same program, using the collective-style operation would
be:
Process A Process B
--------- ---------
store
GET_GO GET_ARG
GET_ARG
GET_ARG
GET_GO
I think that these are better than the barrier-only model, because they remove
the overloading of the functionality of BARRIER. This can help the user to
understand their program (instead of wondering about the effect of a BARRIER
at some point in the program), and it can help the implementor to optimize
the operation (I expect) by knowing the precise role of the BARRIER.
There may be times, however, when the target process could do other work while
waiting for the operation to finish, as it could with the barrier-only style.
This can be accomodated by allowing a non-blocking version of PUT_GO and
GET_GO (i.e. IPUT_GO and IGET_GO).
-Dave
P.S. Some of these names were used to avoid confusion. I would prefer
calling PUT_GO and GET_GO by their original names -- PUT and GET -- in
a formal proposal.
===============================================================================
David C. DiNucci | MRJ, Inc., Rsrch Scntst |USMail: NASA Ames Rsrch Ctr
dinucci@nas.nasa.gov| NAS (Num. Aerospace Sim.)| M/S T27A-2
(415)604-4430 | Parallel Tools Group | Moffett Field, CA 94035