[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: Generalized Request Progress
This example seems compelling to me, though I think Marc missed one
critical word in his conclusion...
The standard does NOT need any fixing, but many implementations do.
(the NOT was previously omitted :-)
-- Jim
James Cownie <james.h.cownie@xxxxxxxxx>
SSG/DPD/PAT
Tel: +44 117 9071438
> -----Original Message-----
> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx]
On
> Behalf Of Marc Snir
> Sent: 03 September 2007 03:49
> To: mpi-21@xxxxxxxxxxxxx
> Cc: lums@xxxxxxxxxxxxxx; dgregor@xxxxxxxxxxxxxx; 'Robert Latham';
> rross@xxxxxxxxxxx
> Subject: Re: Generalized Request Progress
>
> The problem occurs with nonblocking send-receives as well. IMHO, one
> cannot implement MPI without a progress engine, either threads or
> interrupts or offload engine. Of course, most implementations are
> noncompliant on that account.
>
> To be more specific, my understanding of the MPI standard is that in
> the following example:
>
> MPI_Init(...);
> ...
> MPI_Comm_dup(MPI_COMM_WORLD, &comm);
> MPI_Comm_rank(MPI_Comm_World, &myrank);
> switch(myrank) {
> case 0: {
> foo(); /* do some long computation
> MPI_Send(a, large_number, MPI_REAL, 1, tag, MPI_COMM_WORLD);
> MPI_Send(b, 1, MPI_REAL, 2, tag, comm);
> }
> case 1: {
> MPI_Irecv(a, large_number, MPI_REAL, MPI_ANY_SOURCE, tag,
> MPI_COMM_WORLD);
> while(1) ;
> }
> case 2 : {
> MPI_Recv(b, 1, MPI_REAL, 0, tag, comm);
> MPI_Abort(MPI_COMM_WORLD, 666);
> }
> MPI_Finalize();
>
> The program should abort, and should not be stuck in an infinite
> loop: the receive on process 1 should make progress even though no
> MPI call occurs on that process after the receive. Therefore, the
> first send on process 0 should complete, the second send should
> start, and the receive on process 2 should complete.
>
> In this view, there is no difference between generalized requests and
> send or receive requests; it is just that it is much easier to run
> into problems with progress of generalized requests. The standard
> does need any fixing, but many implementations do.
>
> Marc Snir
> Interim Director, Illinois Informatics Initiative
> University of Illinois at Urbana Champaign
> Library and Information Systems Building, MC-493
> 501 E. Daniel St, Room 123
> Champaign, IL 61820-6211
> Tel: (217) 244 6568 Fax: (217) 244 3302
> www.cs.uiuc.edu/homes/snir
>
>
>
>
>
> On Aug 30, 2007, at 3:55 PM, Rajeev Thakur wrote:
>
> > Torsten,
> > Interesting note! For this very reason we have proposed some
> > extensions to the generalized request interface in a paper to be
> > presented
> > at Euro PVM/MPI next month. We face the same problem in
> > implementing the
> > MPI-IO nonblocking I/O functions using generalized requests. You
> > can find
> > the paper at www.mcs.anl.gov/~thakur/papers/grequest-redesign.pdf.
> >
> > Regards,
> > Rajeev
> >
> >> -----Original Message-----
> >> From: owner-mpi-21@xxxxxxxxxxxxx
> >> [mailto:owner-mpi-21@xxxxxxxxxxxxx] On Behalf Of Torsten Hoefler
> >> Sent: Thursday, August 30, 2007 1:52 PM
> >> To: mpi-21@xxxxxxxxxxxxx
> >> Cc: lums@xxxxxxxxxxxxxx; dgregor@xxxxxxxxxxxxxx
> >> Subject: Generalized Request Progress
> >>
> >> Hello,
> >> we (Douglas Gregor and me) tried to port LibNBC [1] to use MPI
> >> generalized requests to enable MPI_Wait{all|some}() for a specific
> >> application scenario.
> >>
> >> LibNBC uses an internal state machine to represent the stage
> >> (round) of
> >> the collective operation. A progress function (in this case
> >> NBC_Test())
> >> needs to be called to ensure internal progress (of the collective
> >> operation).
> >>
> >> We realized that there is no way to ensure progress with
generalized
> >> requests (without threads) because there is no function that is
> >> called
> >> everytime the user calls test or wait on the request (the query_fn
is
> >> only called after the request is finished). This seems to enforce a
> >> threaded implementation of any non-blocking operation that uses
> >> generalized requests. This scheme seems to be suboptimal in some
> >> use-cases. First, not all parallel environments support threads
> >> (e.g.,
> >> Catamount) and second, we show in [2] that, at least for some
> >> applications, a non-threaded implementation may perform better than
a
> >> threaded implementation.
> >>
> >> We would like to add some way to progress the operation
> >> without threads.
> >> We consider this issue as a bug that renders the current interface
> >> useless for several use-cases and it is easy to fix (thus,
> >> we'd like to
> >> see a fix in MPI 2.1). The possible two fixes seem to be
> >> reasonable and
> >> easy to implement:
> >>
> >> 1) change the behavior of query_fn such as that it is called
> >> every time
> >> when the associated request is tested or waited on (spinning
> >> in case of
> >> wait) - this may be suboptimal, because it could break
> >> existing code and
> >> changes the defined behavior
> >>
> >> 2) add a new function "MPI_Grequest_start_progress(progress_fn,
> >> query_fn, free_fn, cancel_fn, data, request)" that defines the
passed
> >> progress function to be called when the associated request is
> >> tested or
> >> waited on.
> >>
> >> Any comments? We are of course open for other suggestions and
> >> discussions!
> >>
> >> Thanks & Best,
> >> Torsten
> >>
> >> [1]: http://www.unixer.de/research/nbcoll/libnbc/
> >> [2]: T. Hoefler and P. Kambadur and R. L. Graham and G. Shipman and
> >> A. Lumsdaine: A Case for Standard Non-Blocking Collective
> >> Operations
> >> (accepted for publication at the EuroPVM/MPI 2007)
> >>
> >> --
> >> bash$ :(){ :|:&};: --------------------- http://www.unixer.de/
-----
> >> Indiana University | http://www.indiana.edu
> >> Open Systems Lab | http://osl.iu.edu/
> >> 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA
> >> Lindley Hall Room 135 | +01 (812) 855-3608
> >>
> >>
> >
---------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.