Alok
At 04:33 PM 1/30/97 PST, you wrote:
>
>I am interested in discussing if we should have non-blocking collective
>(asynchronous) functions as part of I/O. I am not interested in pursuing
>non-blocking collective calls for communication; I wish only to discuss
>these functions with regard to I/O.
>
>I realize that time is short and that we'll need a proposal by Feb 7 if we
>are able to agree that such calls are both technically possible and useful.
>To expedite the discussion, I propose that we first focus our discussion to
>whether or not such I/O calls are technically feasible (i.e. we are unable
>to find a reason that would preclude a reasonably efficient implementation).
>I believe that there is utility in such I/O calls -- we can cover
>the necessary discussion on whether or not these calls are beneficial
>once we have first made a positive finding on if they are possible.
>
>So, in trying to determine if there are technical reasons to prohibit
>non-blocking collective (asynchronous) I/O functions, let me begin
>this discussion by listing the difficulties associated with such calls.
>This list is probably incomplete, please add what I am missing.
>After the list you will find my initial response to the stated problem.
>
>1. The List: (Difficulties associated with non-blocking collective calls)
>
> 1.1) May require large amounts of state to be passed around.
>
> 1.2) Fortran 90 array slices may be implemented on stack and therefore
> may cause inappropriate behavior. For example, in the following
> code snippet, a compiler could make a new copy of 50 elements from
> array x (every other element) onto the stack. However, this stack
> may only be valid inside the scope of foo. (While this kind of
> behavior is similar to C programmers misusing automatic variables,
> the difference is that a C program would be erroneous but the Fortran
> program is valid.)
>
> program example1
> real x(100)
> call foo(x(::2))
> <the non-blocking call from subroutine foo pays off here>
> end
>
> subroutine foo(yy)
> real yy(*)
> <perform some non-blocking call with array yy here>
> return
> end
>
> 1.3) May place additional constraints on the progress_fn. For example, a
> multi-phase function like barrier (which requires some kind of reduce
> comprised of multiple messages from a given node) would need to
> progress until a "sane" state.
>
> 1.4) Would allow a non-blocking read into a variable ``i'' which the
> compiler could actually have in a register. The compiler could
> later flush its (old) value from the register to ``i'' and overwrite
> the result of the non-blocking read.
>
> 1.5) Increases potential for deadlock from interleaved collective
> operations.
>
> 1.6) Matching collective operations becomes a concern.
>
> 1.7) What are the rules for the progress engine? When do reads and
> writes happen?
>
> 1.8) (I must be very vague here because I do not understand the issue.
> Would some kind soul please elaborate.)
> I believe I heard allusion to some kind of issue with the completion
> operations MPI_{TEST|WAIT}{ANY|SOME|ALL} at the recent MPI-forum
> meeting earlier this month. If I understand the concern correctly,
> only MPI_Wait can reasonably be used with non-blocking collective
> operations. Someone will have to fill in the details here.
>
>2. Discussion of the list
>
> Since difficulties 1.1 through 1.4 apply to non-blocking point-to-point
> communications in MPI 1.1, I'll assume that they have been determined
> to not be prohibitive there. Therefore, for non-blocking collective I/O
> (possibly implemented as a layered library), I think we can say that
> problems 1.1 through 1.4 reduce to "can we get Generalized Requests right"
> which is an ongoing topic in the mpi-external mailing-list.
>
> Regarding 1.5 and 1.6, it seems like maintaining a sequence number
> with the request state would be sufficient (i.e. read 16 on node xx
> must match read 16 on node yy, and so forth). Assuming that the
> implementation is able to match the operation orderings, any remaining
> potential for problems is a program correctness issue.
>
> Regarding 1.7, is this only a problem for overlapping reads/writes?
> If so, I think any method which provides the following two requirements
> should work: a) an ordering of the data accesses; b) atomic reads/writes
> of the entire buffer. Requirement a) can be met through either using
> globally synchronized timestamps or using the order of messages received
> at a single location.
>
> BOTTOM LINE: I think we have reasonable ways to address 1.1 through 1.6.
> For other problems, someone else will have to provide some further
> description of the problem.
>
>-terry
>
>