Re: Explicit Job Scheduler Interface

Suresh Damodaran-Kamal (suresh@lanl.gov)
Tue, 29 Aug 1995 18:22:37 -0600

> From mpi-dynamic-human@mcs.anl.gov Tue Aug 29 17:16:30 1995
> Date: Tue, 29 Aug 1995 16:13:34 -0700
> From: wcs@nas.nasa.gov (William C. Saphir)
>
> > Date: Tue, 29 Aug 1995 15:13:08 -0600
> > From: Suresh Damodaran-Kamal <suresh@lanl.gov>
> > To: wcs
> > Subject: Explicit Job Scheduler Interface
> > Cc: mpi-dynamic@anl.gov, mpi-external@anl.gov
> >
> > Chapter 5 in MPI-2 (on dynamic process management) does not
> > seem to have an example on the interface with external (to MPI)
> > job schedulers ( load balancers and debuggers).
> >
> > Is is possible to do such interfacing using the current form of
> > MPI-2? (It didn't look like such provisions exist after reading the chapter-
> > correct me if I am wrong).
>
> Correct. No interface is specified. It has been pushed off to the
> realm of "implementation issue". Clearly there will need to be some
> interaction between MPI implementations and external resource/process
> managers, but I don't think there are any fundamental barriers to
> interaction.
>

I agree that there are no fundamental barriers to such interaction.
The interface currently in the proposal seems to be adequate (in abstract)
for interfacing with resource managers. However, I think an interface
has to be specified with external process managers (which do job scheduling,
and also interface with resource managers).
In particular, the mechanism of delegation of process creation
capabilities to the job scheduler (which can be also a distributed
debugger, or a load balancer).

> In the area of process management, there are probably no big
> problems.

MPI_process_create() creates a process using the Resource that is passed
to it. However, there is no way for say, a distributed debugger, to ask
the process manager to delegate the responsibility of actually creating
the process to it. PVM handles it with a pvm_reg_tasker() call.
This facility is really needed for distributed debugging that does
not alter the source code of the users and MPI.
(Actually, our debugger HPD uses this call with PVM, also TotalView from BBN
uses this call).

>
> The interaction with debuggers is another issue, which we
> haven't considered in the dynamic process management chapter.
> My first impression is that this is outside MPI, and is
> an issue between the debugger and the process manager (as
> defined in chap 5). The debugger presumably has to intervene
> when processes are created. From the point of view of MPI,
> process creation itself is entirely outside MPI. So for instance,
> MPI might know the process creator as a script called
> "create_process". Normally this script might contain "rsh"
> but when debugging is turned on, this script would contain
> "dbx" as well. But MPI wouldn't need to know.

It is not clear how one could tell MPI to use the "create_process"
script for starting the processes. If this issue is really outside
of MPI, it will severely limit portability and heterogeneous operation
of load balancers and distributed debuggers. A clean mechanism
specified in MPI will really be neat and useful.