14.2. Generalized Requests

PreviousUpNext
Up: External Interfaces Next: Examples Previous: Introduction

The goal of generalized requests is to allow users to define new nonblocking operations. Such an outstanding nonblocking operation is represented by a (generalized) request. A fundamental property of nonblocking operations is that progress toward the completion of this operation occurs asynchronously, i.e., concurrently with normal program execution. Typically, this requires execution of code concurrently with the execution of the user code, e.g., in a separate thread or in a signal handler. Operating systems provide a variety of mechanisms in support of concurrent execution. MPI does not attempt to standardize or to replace these mechanisms: it is assumed programmers who wish to define new asynchronous operations will use the mechanisms provided by the underlying operating system. Thus, the calls in this section only provide a means for defining the effect of MPI calls such as MPI_WAIT or MPI_CANCEL when they apply to generalized requests, and for signaling to MPI the completion of a generalized operation.


Rationale.

It is tempting to also define an MPI standard mechanism for achieving concurrent execution of user-defined nonblocking operations. However, it is difficult to define such a mechanism without consideration of the specific mechanisms used in the operating system. The Forum feels that concurrency mechanisms are a proper part of the underlying operating system and should not be standardized by MPI; the MPI standard should only deal with the interaction of such mechanisms with MPI. ( End of rationale.)
For a regular request, the operation associated with the request is performed by the MPI implementation, and the operation completes without intervention by the application. For a generalized request, the operation associated with the request is performed by the application; therefore, the application must notify MPI through a call to MPI_GREQUEST_COMPLETE when the operation completes. MPI maintains the ``completion'' status of generalized requests. Any other request state has to be maintained by the user.

A new generalized request is started with

MPI_GREQUEST_START(query_fn, free_fn, cancel_fn, extra_state, request)
IN query_fncallback function invoked when request status is queried (function)
IN free_fncallback function invoked when request is freed (function)
IN cancel_fncallback function invoked when request is cancelled (function)
IN extra_stateextra state
OUT requestgeneralized request (handle)
C binding
int MPI_Grequest_start(MPI_Grequest_query_function *query_fn, MPI_Grequest_free_function *free_fn, MPI_Grequest_cancel_function *cancel_fn, void *extra_state, MPI_Request *request)
Fortran 2008 binding
MPI_Grequest_start(query_fn, free_fn, cancel_fn, extra_state, request, ierror)

PROCEDURE(MPI_Grequest_query_function) :: query_fn
PROCEDURE(MPI_Grequest_free_function) :: free_fn
PROCEDURE(MPI_Grequest_cancel_function) :: cancel_fn
INTEGER(KIND=MPI_ADDRESS_KIND), INTENT(IN) :: extra_state
TYPE(MPI_Request), INTENT(OUT) :: request
INTEGER, OPTIONAL, INTENT(OUT) :: ierror
Fortran binding
MPI_GREQUEST_START(QUERY_FN, FREE_FN, CANCEL_FN, EXTRA_STATE, REQUEST, IERROR)

EXTERNAL QUERY_FN, FREE_FN, CANCEL_FN
INTEGER(KIND=MPI_ADDRESS_KIND) EXTRA_STATE
INTEGER REQUEST, IERROR


Advice to users.

Note that a generalized request is of the same type as regular requests, in C and Fortran. ( End of advice to users.)
The call starts a generalized request and returns a handle to it in request.

The syntax and meaning of the callback functions are listed below. All callback functions are passed the extra_state argument that was associated with the request by the starting call MPI_GREQUEST_START; extra_state can be used to maintain user-defined state for the request.

In C, the query procedure is

typedef int MPI_Grequest_query_function(void *extra_state, MPI_Status *status);


in Fortran with the mpi_f08 module

ABSTRACT INTERFACE
    SUBROUTINE MPI_Grequest_query_function(extra_state, status, ierror)

INTEGER(KIND=MPI_ADDRESS_KIND) :: extra_state
TYPE(MPI_Status) :: status
INTEGER :: ierror


in Fortran with the mpi module and (deprecated) mpif.h include file

SUBROUTINE GREQUEST_QUERY_FUNCTION(EXTRA_STATE, STATUS, IERROR)

INTEGER(KIND=MPI_ADDRESS_KIND) EXTRA_STATE
INTEGER STATUS(MPI_STATUS_SIZE), IERROR


The query_fn function computes the status that should be returned for the generalized request. The status also includes information about successful/unsuccessful cancellation of the request (result to be returned by MPI_TEST_CANCELLED).

The query_fn callback is invoked by the MPI_{WAIT|TEST}{ANY|SOME|ALL} call that completed the generalized request associated with this callback. The callback function is also invoked by calls to MPI_REQUEST_GET_STATUS, if the request is complete when the call occurs. In both cases, the callback is passed a reference to the corresponding status variable passed by the user to the MPI call; the status set by the callback function is returned by the MPI call. If the user provided MPI_STATUS_IGNORE or MPI_STATUSES_IGNORE to the MPI procedure that causes query_fn to be called, then MPI will pass a valid status object to query_fn, and this status will be ignored upon return of the callback function. Note that query_fn is invoked only after MPI_GREQUEST_COMPLETE is called on the request; it may be invoked several times for the same generalized request, e.g., if the user calls MPI_REQUEST_GET_STATUS several times for this request. Note also that a call to MPI_{WAIT|TEST}{SOME|ALL} may cause multiple invocations of query_fn callback functions, one for each generalized request that is completed by the MPI call. The order of these invocations is not specified by MPI.

In C, the free procedures is

typedef int MPI_Grequest_free_function(void *extra_state);


in Fortran with the mpi_f08 module

ABSTRACT INTERFACE
    SUBROUTINE MPI_Grequest_free_function(extra_state, ierror)

INTEGER(KIND=MPI_ADDRESS_KIND) :: extra_state
INTEGER :: ierror


in Fortran with the mpi module and (deprecated) mpif.h include file

SUBROUTINE GREQUEST_FREE_FUNCTION(EXTRA_STATE, IERROR)

INTEGER(KIND=MPI_ADDRESS_KIND) EXTRA_STATE
INTEGER IERROR


The free_fn function is invoked to clean up user-allocated resources when the generalized request is freed.

The free_fn callback is invoked by the MPI_{WAIT|TEST}{ANY|SOME|ALL} call that completed the generalized request associated with this callback. free_fn is invoked after the call to query_fn for the same request. However, if the MPI call completed multiple generalized requests, the order in which free_fn callback functions are invoked is not specified by MPI.

The free_fn callback is also invoked for generalized requests that are freed by a call to MPI_REQUEST_FREE (no call to MPI_{WAIT|TEST}{ANY|SOME|ALL} will occur for such a request). In this case, the callback function will be called either in the MPI call MPI_REQUEST_FREE(request), or in the MPI call MPI_GREQUEST_COMPLETE(request), whichever happens last, i.e., in this case the actual freeing code is executed as soon as both calls MPI_REQUEST_FREE and MPI_GREQUEST_COMPLETE have occurred. The request is not deallocated until after free_fn completes. Note that free_fn will be invoked only once per request by a correct program.


Advice to users.

Calling MPI_REQUEST_FREE(request) will cause the request handle to be set to MPI_REQUEST_NULL. This handle to the generalized request is no longer valid. However, user copies of this handle are valid until after free_fn completes since MPI does not deallocate the object until then. Since free_fn is not called until after MPI_GREQUEST_COMPLETE, the user copy of the handle can be used to make this call. Users should note that MPI will deallocate the object after free_fn executes. At this point, user copies of the request handle no longer point to a valid request. MPI will not set user copies to MPI_REQUEST_NULL in this case, so it is up to the user to avoid accessing this stale handle. This is a special case in which MPI defers deallocating the object until a later time that is known by the user. ( End of advice to users.)
In C, the cancel procedure is

typedef int MPI_Grequest_cancel_function(void *extra_state, int complete);


in Fortran with the mpi_f08 module

ABSTRACT INTERFACE
    SUBROUTINE MPI_Grequest_cancel_function(extra_state, complete, ierror)

INTEGER(KIND=MPI_ADDRESS_KIND) :: extra_state
LOGICAL :: complete
INTEGER :: ierror


in Fortran with the mpi module and (deprecated) mpif.h include file

SUBROUTINE GREQUEST_CANCEL_FUNCTION(EXTRA_STATE, COMPLETE, IERROR)

INTEGER(KIND=MPI_ADDRESS_KIND) EXTRA_STATE
LOGICAL COMPLETE
INTEGER IERROR

The cancel_fn function is invoked to start the cancelation of a generalized request. It is called by MPI_CANCEL(request). MPI passes complete = true to the callback function if MPI_GREQUEST_COMPLETE was already called on the request, and complete = false otherwise.

All callback functions return an error code. The code is passed back and dealt with as appropriate for the error code by the MPI procedure that invoked the callback function. For example, if error codes are returned then the error code returned by the callback function will be returned by the MPI procedure that invoked the callback function. In the case of an MPI_{WAIT|TEST}{ANY} call that invokes both query_fn and free_fn, the MPI call will return the error code returned by the last callback, namely free_fn. If one or more of the requests in a call to MPI_{WAIT|TEST}{SOME|ALL} failed, then the MPI call will return MPI_ERR_IN_STATUS. In such a case, if the MPI call was passed an array of statuses, then MPI will return in each of the statuses that correspond to a completed generalized request the error code returned by the corresponding invocation of its free_fn callback function. However, if the MPI procedure was passed MPI_STATUSES_IGNORE, then the individual error codes returned by each callback functions will be lost.


Advice to users.

query_fn must not set the error field of status since query_fn may be called by MPI_WAIT or MPI_TEST, in which case the error field of status should not change. The MPI library knows the ``context'' in which query_fn is invoked and can decide correctly when to put the returned error code in the error field of status. ( End of advice to users.)

MPI_GREQUEST_COMPLETE(request)
INOUT requestgeneralized request (handle)
C binding
int MPI_Grequest_complete(MPI_Request request)
Fortran 2008 binding
MPI_Grequest_complete(request, ierror)

TYPE(MPI_Request), INTENT(IN) :: request
INTEGER, OPTIONAL, INTENT(OUT) :: ierror
Fortran binding
MPI_GREQUEST_COMPLETE(REQUEST, IERROR)

INTEGER REQUEST, IERROR

The call informs MPI that the operations represented by the generalized request request are complete (see definitions in Section Semantic Terms). A call to MPI_WAIT(request, status) will return and a call to MPI_TEST(request, flag, status) will return flag = true only after a call to MPI_GREQUEST_COMPLETE has declared that these operations are complete.

MPI imposes no restrictions on the code executed by the callback functions. However, new nonblocking operations should be defined so that the general semantic rules about MPI calls such as MPI_TEST, MPI_REQUEST_FREE, or MPI_CANCEL still hold. For example, these calls are supposed to be local and nonblocking. Therefore, the callback functions query_fn, free_fn, or cancel_fn should invoke blocking MPI communication calls only if the context is such that these calls are guaranteed to return in finite time. Once MPI_CANCEL is invoked, the cancelled operation should complete in finite time, irrespective of the state of other MPI processes (the operation has acquired ``local'' semantics). It should either succeed, or fail without side-effects. The user should guarantee these same properties for newly defined operations.


Advice to implementors.

A call to MPI_GREQUEST_COMPLETE may unblock a blocked user process/thread. The MPI library should ensure that the blocked user computation will resume. ( End of advice to implementors.)


PreviousUpNext
Up: External Interfaces Next: Examples Previous: Introduction


Return to MPI-4.1 Standard Index
Return to MPI Forum Home Page

(Unofficial) MPI-4.1 of November 2, 2023
HTML Generated on November 19, 2023