Re: dynamic counter-proposal

Greg Burns (gdburns@tbag.osc.edu)
Fri, 10 May 1996 16:18:27 -0400 (EDT)

Bill G. writes:
>I am concerned about the implementation complexity of the spawn/attach
>approach in the case of an MPP with an interconnect that must be reconfigured
>in order to provide high-performance connections. I am ignoring the
>user-hostile approach of only permitting "spawn" into a preallocated
>partition. This interconnect might be a physical device or the way an OS
>keeps track of seperate executable objectss (I'm avoiding threads/processes
>deliberately).

You must be thinking of a specific example. It would help my understanding.
My only related experience is dealing with transputer link switches where
they would be set during the application start-up procedure to reflect,
as much as possible, the communication topology. In this context the
reconfiguration would only be done once after the spawn step, for the
real performance issue was getting the application topology to fit
the machine's. The spawn messages could go out on any network - we
did not care.

>2) There are other issues that are likely to need to know if the spawn is
>creating processes that are an extension of an existing set of a new set.
>For example, the user may want to gang or co-schedule the new processes with
>the existing set.

>3) An interesting model that is popping up on some shared memory machines is
>"threads with private memory". To the MPI user, this looks like an MPI
>process (since it has its own memory space); to the implementation, it offers
>major advantages in performance (both with shared memory and intelligent
>scheduling). If you are doing a spawn of the same executable for the purposes
>of expanding your communicator, you can create a new such "thread"; if you are
>spawning to create a new independent process, you want to fork/exec. It is
>too late to do this when the the attach/detach occurs. Technically, it is
>possible to change the operating system to allow threads to be stripped off
>into another process, but I can't see why it should be done just to support
>MPI. Also note that in both cases the user is likely to use the NULL info
>argument, making it impossible to know what to do until it is too late.

Bill G. illuminates the differing gene that I glossed over. In the current
draft you have that extra bit of information at spawn time (independent
or not). In the new proposal you do not. You may need to pass this
1-bit to a single function underlying spawn function in the OS. Then he
drives a wedge into that crack with two realistic examples. One has to
judge how worrisome they are. To fix it, you would have to add another
flag into MPI_Spawn, new proposal, for INDEP. We already have
the MPI/NONMPI flag to cope with other realistic worries, brought forth
in early private discussions. Then this starts to look a lot like
an earlier version of the current draft, as Rusty reminds us.
[The difference is that the separate attach/detach gets rid of the
_multiple stuff.]

I mentioned I don't like NONMPI and this discussion reminds me of old
feelings against independent spawning. Who needs it? Is it worth the
trouble? Get rid of them both and you don't need a separate attach/detach.
Hey, you know what? This is now exactly the spawn in LAM 6.0.
No surprise, I guess. It's just a long time since we did that part of
the development and I figured it would have zero hope in the Forum, so
I put it out of my mind.

Al Geist strongly believes that there should be a flag argument in
MPI_Spawn for future contingencies, even if no bits are currently
defined. I'm convinced.

--
Greg