OK, it changes the one thing that was probably the hardest to design
and make acceptable to most people. But starting over?, major
change in direction?, getting it all wrong?, I don't see that at all.
A pleasant (but not huge :-) stroke of artistic inspiration is what I see.
to the old one, except after getting the application started, instead
of putting in a big comment introducing the communicator synchronization
phase, I put a return statement and put the code in another function
(MPI_Child_attach). I'm serious. The new trick is that by allowing
the user to play with groups, we can lose the _multiple variants of
MPI_Spawn. By separating process creation and communicator building,
we can lose the _nonmpi variants. Instead of having all combinations
of three features in the API, we give the user the building blocks.
To reduce code duplication, I would expect implementations to do
exactly the same thing internally.
>We have
>begun implementing it, and it works fine. No daemons or other third parties
>are required, and it is easy to see how to interface it to process managers
>like LSF.
We have implemented and distributed a spawn capability as part of
LAM 6.0 that has exactly the same communicator usage as is proposed by
the current draft (and unchanged in the new proposal). I forsee no
problems breaking it into separate creation and communicator hookup
functions. In fact, I think it is trivial.
Please explain exactly how the process manager complicates this division.
I would not expect third party process managers to be the synchronization
point for comm_world construction, since they know nothing about MPI.
Thus I don't anticipate a problem with them doing the actual creation
and the MPI parent processes doing the communicator construction.
>(Remember that workstation networks are not the only environment, and that an
>implementation may not have access to the internals of the process manager.)
>For this to work, it is critical that the setting up of the communicator be
>done in conjunction with the creation of the processes.
So you are asserting that environment portability will not work without
combining the two steps in a single MPI function? I take this
seriously since you are travelling down this road, but please explain
the details.
>Separating them
>constrains the implementation so that it can no longer do everything at once
>at a time when it has the participation of all the other processes, and can
>devise a strategy for either succeeding (delivering a communicator) or failing
>as cleanly as it can figure out how to do.
Are you referring to Bill's example of 6 processes created cleanly, but
only 5 show up for comm_world building? In a single call, we might be
able to use minprocs to come back with a communicator of 5 processes and
declare all to be well. In two calls, the user is stuck with this
wounded group and we would have to fail the whole comm_world construction
function. Maybe not. We could consume the group handle and return
an inter-communicator with 5 processes. Anyways, I am unmoved by
this case-based problem because I don't like minprocs, but that is
another issue.
>Even just from the user's point of view,
>
> MPI_Spawn( ... , newcomm)
>
>is more convenient than
>
> MPI_Spawn( ... , group) /*introduction of spurious new MPI object */
> MPI_Child_Attach( ... group, newcomm)
I agree with this, but we are buying a lot in return, in my judgement.
In another judgement call, I don't think spawning non-MPI processes has
a value to the overall MPI mission that justifies its additional
complexity in the standard. Without it, we could also remove the
MPI/NONMPI flag from the new proposal's MPI_Spawn. Yes, I know this
issue was already decided in the other direction with a straw vote
although I know at least three implementors that would prefer to see it go.
-- Greg