[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Generalized Request Progress

The problem occurs with nonblocking send-receives as well. IMHO, one cannot implement MPI without a progress engine, either threads or interrupts or offload engine. Of course, most implementations are noncompliant on that account.

To be more specific, my understanding of the MPI standard is that in the following example:

MPI_Init(...);
...
MPI_Comm_dup(MPI_COMM_WORLD, &comm);
MPI_Comm_rank(MPI_Comm_World, &myrank);
switch(myrank) {
case 0: {
foo(); /* do some long computation
MPI_Send(a, large_number, MPI_REAL, 1, tag, MPI_COMM_WORLD);
MPI_Send(b, 1, MPI_REAL, 2, tag, comm);
}
case 1: {
MPI_Irecv(a, large_number, MPI_REAL, MPI_ANY_SOURCE, tag, MPI_COMM_WORLD);
while(1) ;
}
case 2 : {
MPI_Recv(b, 1, MPI_REAL, 0, tag, comm);
MPI_Abort(MPI_COMM_WORLD, 666);
}
MPI_Finalize();

The program should abort, and should not be stuck in an infinite loop: the receive on process 1 should make progress even though no MPI call occurs on that process after the receive. Therefore, the first send on process 0 should complete, the second send should start, and the receive on process 2 should complete.

In this view, there is no difference between generalized requests and send or receive requests; it is just that it is much easier to run into problems with progress of generalized requests. The standard does need any fixing, but many implementations do.

Marc Snir
Interim Director, Illinois Informatics Initiative
University of Illinois at Urbana Champaign
Library and Information Systems Building, MC-493
501 E. Daniel St, Room 123
Champaign, IL 61820-6211
Tel: (217) 244 6568 Fax: (217) 244 3302
www.cs.uiuc.edu/homes/snir





On Aug 30, 2007, at 3:55 PM, Rajeev Thakur wrote:

Torsten,
Interesting note! For this very reason we have proposed some
extensions to the generalized request interface in a paper to be presented
at Euro PVM/MPI next month. We face the same problem in implementing the
MPI-IO nonblocking I/O functions using generalized requests. You can find
the paper at www.mcs.anl.gov/~thakur/papers/grequest-redesign.pdf.

Regards,
Rajeev

-----Original Message-----
From: owner-mpi-21@xxxxxxxxxxxxx
[mailto:owner-mpi-21@xxxxxxxxxxxxx] On Behalf Of Torsten Hoefler
Sent: Thursday, August 30, 2007 1:52 PM
To: mpi-21@xxxxxxxxxxxxx
Cc: lums@xxxxxxxxxxxxxx; dgregor@xxxxxxxxxxxxxx
Subject: Generalized Request Progress

Hello,
we (Douglas Gregor and me) tried to port LibNBC [1] to use MPI
generalized requests to enable MPI_Wait{all|some}() for a specific
application scenario.

LibNBC uses an internal state machine to represent the stage
(round) of
the collective operation. A progress function (in this case
NBC_Test())
needs to be called to ensure internal progress (of the collective
operation).

We realized that there is no way to ensure progress with generalized
requests (without threads) because there is no function that is called
everytime the user calls test or wait on the request (the query_fn is
only called after the request is finished). This seems to enforce a
threaded implementation of any non-blocking operation that uses
generalized requests. This scheme seems to be suboptimal in some
use-cases. First, not all parallel environments support threads (e.g.,
Catamount) and second, we show in [2] that, at least for some
applications, a non-threaded implementation may perform better than a
threaded implementation.

We would like to add some way to progress the operation
without threads.
We consider this issue as a bug that renders the current interface
useless for several use-cases and it is easy to fix (thus,
we'd like to
see a fix in MPI 2.1). The possible two fixes seem to be
reasonable and
easy to implement:

1) change the behavior of query_fn such as that it is called
every time
when the associated request is tested or waited on (spinning
in case of
wait) - this may be suboptimal, because it could break
existing code and
changes the defined behavior

2) add a new function "MPI_Grequest_start_progress(progress_fn,
query_fn, free_fn, cancel_fn, data, request)" that defines the passed
progress function to be called when the associated request is
tested or
waited on.

Any comments? We are of course open for other suggestions and
discussions!

Thanks & Best,
  Torsten

[1]: http://www.unixer.de/research/nbcoll/libnbc/
[2]: T. Hoefler and P. Kambadur and R. L. Graham and G. Shipman and
A. Lumsdaine: A Case for Standard Non-Blocking Collective Operations
(accepted for publication at the EuroPVM/MPI 2007)

--
 bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ -----
Indiana University    | http://www.indiana.edu
Open Systems Lab      | http://osl.iu.edu/
150 S. Woodlawn Ave.  | Bloomington, IN, 474045-7104 | USA
Lindley Hall Room 135 | +01 (812) 855-3608