> ... What you seem to do here is just remove demand for progress.
I agree, my rule can be expressed by an extremely restricted
progress rule:
A non-blocking collective operation must make progress only while
waiting with MPI_WAIT for that operation.
(Formally this is no change of MPI-1 because MPI-1 did not
specify the progress of non-blocking collective operations.)
> If this is so, I suggest that any time any blocking collective is called
> over a group G, then the MPI system may progress non-blocking
> collective operations over G (and possibly all proper subsets, tbd).
I made this extrem restriction because I looked at the following
example
1 2 3
Ibarrier(rq1a) ---- Ibarrier(rq2a)
Ibarrier(rq2b) ---- Ibarrier(rq3b)
Ibarrier(rq1c) ------------------------ Ibarrier(rq3c)
WAIT_ALL(rq1a&rq1c) WAIT_ALL(rq2a&rq2b) WAIT_ALL(rq3b&rq3c)
and I wanted to allow that implementor use their hardware barrier
and therefore I thought:
If proc. 2 would first call internally rq2b-barrier
and proc. 1 first rq1a-barrier then we would have a deadlock.
There is no way for the MPI impl. to see which request should be
done first.
If you require progress while blocking collective operations
then you have the same problem as if you allow MPI_WAIT_ALL:
Ibarrier(rq1a) ---- Ibarrier(rq2a)
Ibarrier(rq2b) ---- Ibarrier(rq3b)
Ibarrier(rq1c) ------------------------ Ibarrier(rq3c)
Barrier ----------- Barrier ----------- Barrier
Wait(rq1a) Wait(rq2a) Wait(rq3b)
Wait(rq1c) Wait(rq2b) Wait(rq3c)
> I think we were expecting progress in the non-blocking collectives in the
> same symmetry [per Marc] we would get from Isend or Irecv type operations.
I think, there are three requirements
- user want to have non-blocking reduce, ... to overlap
computing and communication!
- some implementors see that they can support the user
in this way!
- some other implementors do not want to have non-blocking reduce
if they cannot use there existing efficient blocking implementations!
and until now there was no compromise that can be accepted by
all three groups.
I think, my proposal fulfills all three requirements.
> Weakening the semantics certainly makes the implementations feasible, but
> I expect that it also diminishes usefulness, as it is asserted that
> overlapping the collective operation with other work is what is wanted.
If you look at overlapping of non-blocking collective and point-to-point
you are right, but I think users first want to overlap communication
with computation.
Does my arguments represent the discussion in the forum adequately?
Do you see a weaker restrictive proposal that also fulfills
the third requirement?
Rolf
Rolf Rabenseifner (Computer Center )
Rechenzentrum Universitaet Stuttgart (University of Stuttgart)
Allmandring 30 Phone: ++49 711 6855530
D-70550 Stuttgart 80 FAX: ++49 711 6787626
Germany rabenseifner@rus.uni-stuttgart.de