Re: Marc's new GR section

Parkson Wong (parkson@nas.nasa.gov)
Mon, 3 Mar 1997 13:16:16 -0800 (PST)

>
> Of the three proposals, I prefer version 1 for its simplicity.
> It does require threads, but I'm afraid there may be no option but to
> require threads for generalized requests.
>
> Rajeev
>
> > Date: Tue, 25 Feb 1997 13:37:52 -0600 (CST)
> > From: Steve Huss-Lederman <lederman@cs.wisc.edu>
> >
> > I will overlook the typos and example mistakes for now to focus on
> > what I consider to be the bigger picture. Once we choose a method, we
> > can fix up the text. I think airing our views before the next meeting
> > is important since something will be done at this meeting.
> >
> > From my reading, none of Marc's proposals are portable between
> > threaded and non-threaded MPI implementations. In version 1, you need
> > threads to make progress. In versions 2, you can only use
> > MPI_HANDLER_ENABLE/DISABLE when not on a thread-compliant MPI. They
> > all seem to require the use of the thread locking mechanisms when you
> > are on a threaded MPI and these are not standardized. The thread
> > query functions will allow one to see what is the thread status on
> > this implementation so the method can work in principle. We may need
> > to make the GRs nonportable but I think it should be a concious and up
> > front decision.
> >
> > All the proposals acheive the goal of nonblocking operations where one
> > can do standard MPI operations (wait/test) to check on completion. In
> > that regard they are integrated into MPI. However, as stated above,
> > they are not going to allow one to implement a GR with one piece of
> > portable code. Given this and the one clear use of GR for I/O, I am
> > wondering if the more complex choices (versions 2) are a major
> > improvement over version 1 (Marc prefers version 1). If threads are
> > reasonably fair, then removing progress from MPI is likely to be ok.
> >
> > Steve
>

There are two types of asynchoronous operation, one provides a notification
when the operation is done, the other doesn't (meaning you have to poll
periodically or you need to wait for the operation to complete). MPI-1
is in the later catagory, after a non-blocking operation, you need to do
a TEST or WAIT to find out whether the operation has succeeded.

In the case of generalized request, the same should hold true. However,
the GR proposal require that the external system does an asynchronous
notification GR_MARK_COMPLETE when the request is done, so you have to
spin a thread to do the polling or waiting.

So instead of a querry_fn which is invoked only for a request that has
completed, can we have a wait_fn and test_fn callback function which
are invoked when MPI_WAIT and MPI_TEST is called.

Example:

MPI_Iwrite(...)
{
aio_write(..., handle);
extra_state.handle = handle;
MPI_Gr_start(wait_fn, test_fn, free_fn, cancel_fn, extra_state, request);
}

test_fn(extra_state)
{
aio_test(extra_state.handle);
}

wait_fn(extra_state)
{
aio_wait(extra_state.handle);
}

This assumes the asynchronous I/O modules make progress independently of MPI.
This probably had shown up before, but I forgot why this is a bad idea.

The MPI-II implementation need to be careful with MPI_WAIT_ANY and call
the test function periodically.

We could keep the MPI_Gr_mark_complete for people that wants to use it.
I would suggest modifying to take a status argument, so once this function
is called, only the free_fn need to be invoked.

MPI_Gr_mark_complete(MPI_Gr_request request, MPI_Status status)

In the case that the external subsystem has can't make progress of its
own, we either say we don't support this, or you need threads to guarantee
progress.

-- parkson

-- 
Parkson Wong			Address: Numerical Aerodynamic Simulation
MRJ, Inc.				 NASA Ames Research Center M/S 258-6
Supercomputer Applications Segment	 Moffett Field, CA  94035-1000
e-mail: parkson@nas.nasa.gov	Phone: (415)604-3988	Fax: (415)966-8669