Re: [mpi-21] Proposal: Non-blocking collective operations for MPI-3

[Torsten Hoefler <htor@xxxxxxxxxxxxxx>]

Dear Jesper, all,
> it is in my opinion a very severe (and limiting) constraint that
> you require that non-blocking and blocking collectives must match (p. 5) -
> as we have discussed many times.
Yes, thanks for mentioning this. I (kind of) agree on this - this
proposal is very preliminary to get the discussion started. We
established a working group on "extended collectives" at the forum. We
will decide on things liks this in the group and everybody is free to
join (just let me know if you're interested).

> As argued in my comments on topological collectives, I think non-blocking
> collectives belong in a seperate library on top of MPI, and the task of
> the MPI standard is "just" to make sure that this can be implemented 
> efficiently (they could somehow be associated with MPI as an "auxiliary 
> library" - there might be benefits to that?)
I do also agree on this and am in favor of the "MPI 0.5" or "Mini-MPI" or
however you will call it. But we have to consider that this could be
very confusing. We have the blocking collectives *in* the standard and
the non-blocking variants, that are semantically very close, as a
separate library. And things like MPI/IO that could perfectly be done as
a separate library (cf. ROMIO) are also *in* the standard. That could be
very confusing to users and would seem inconsistent to me. 

I would like to propose a more radical change (open for discussion). I
would vote for a modular MPI, splitting off things like collectives,
non-blocking collectives or MPI/IO in separate libraries. We would have
to think about features in the "core" (or "Mini" or "0.5") MPI that
efficiently enable this modularity. Think about this as a Microkernel-OS
like model. I will work on figuring out those basic interfaces on the
collective side. 

I am not sure (and do not want to propose anything) if the modules
should be standardized like MPI is. But our collectives working group
will come up with a proposal and I am in favor of attaching this as an
official MPI Forum "verified" interface to the standard or whatever. And
I want to mention that it's really easy for vendors to add those
external modules/libraries because reasonably optimized versions are
available under suitable licenses right now (ompi/tuned for blocking
collectives, LibNBC for non-blocking collectives, ROMIO for MPI/IO). 

Best,
  Torsten

-- 
 bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ -----
Indiana University    | http://www.indiana.edu
Open Systems Lab      | http://osl.iu.edu/
150 S. Woodlawn Ave.  | Bloomington, IN, 474045-7104 | USA
Lindley Hall Room 135 | +01 (812) 855-3608