3.4.1. MPI Operations

MPI operation:

An MPI operation is a sequence of steps performed by the MPI library to establish and enable data transfer and/or synchronization. It consists of four stages: initialization, starting, completion, and freeing, and it is implemented as a set of one or more MPI procedures, see Section MPI Procedures.

Initialization

hands over the argument list to the operation but not the content of the data buffers, if any. The specification of an operation may state that array arguments must not be changed until the operation is freed.

Starting

hands over control of the data buffers, if any, to the associated operation.

Note that initiation refers to the combination of the initialization and starting stages.

Completion

returns control of the content of the data buffers and indicates that output buffers and arguments, if any, have been updated.

Note that an MPI operation is complete when the MPI procedure implementing the completion stage returns.

Freeing

returns control of the rest of the argument list (e.g., the data buffer address and array arguments).

Blocking operation:

For a blocking operation, all four stages are combined in a single procedure call (as shown in Figure 1 and defined in Section MPI Procedures).

Figure 1: State transition diagram for blocking operations

Nonblocking operation:

For a nonblocking operation, the initialization and starting stages are combined into a single nonblocking procedure call and the completion and freeing stages are combined into a separate, single procedure call, which can be blocking or nonblocking (as shown in Figure 2 and defined in Section MPI Procedures).

Figure 2: State transition diagram for nonblocking operations

Persistent operation:

For a persistent operation, there is a separate procedure for each of the four stages (as shown in Figure 3 and defined in Section MPI Procedures). Each of these procedures may be blocking or nonblocking.

For a partitioned send operation, an additional call to activate each partition of the send buffer (see Section Communication Initialization and Starting with Partitioning) is required to finish the starting stage. For a partitioned receive operation, before the operation is complete the user is allowed to access a partition of the output buffer after verifying that it has arrived (see Section Communication Completion under Partitioning).

Figure 3: State transition diagram for persistent operations

Active communication and I/O operations are also named pending operations. Note that a pending operation can be a nonblocking or persistent operation that is started and not yet complete (even if the request handle has been freed), or a blocking operation that is not yet complete, such as a receive operation that is waiting for a message to be received.

Additionally, an MPI operation can be collective or noncollective.

Collective operation:

A set of related operations, one per MPI process in a group or groups of MPI processes. For collective operations the completion stage may or may not finish before all processes in the group have started the operation.

Collective MPI operations are also available as blocking, nonblocking, or persistent operations.

Noncollective operation:

Noncollective operations are defined as operations that are not collective.

Enabled:

An MPI operation is enabled at a particular MPI process when all specific semantically-related operations required to guarantee completion at that MPI process have been started.

MPI implementations may include optimizations (for example, automatic buffering) that allow an MPI operation to complete before it is enabled. ( End of rationale.)
Some MPI operations are a priori enabled, i.e., they do not require any other specific semantically-related operation for completion. For example, a buffered send operation completes independently of the related receive operation.

Once an MPI operation is enabled, the operation must eventually complete. An operation may already be enabled before it is started. For example, a receive operation is already enabled if it is started after the matching send operation was started.

Rationale.

The definition of an operation A being enabled is asymmetric: enabled includes that all specific semantically-related operations A'_i required to guarantee completion have been started, but does not include that the operation A itself is already started.

Examples:

• A receive is enabled exactly when the related send is started.
• A standard mode send operation is enabled exactly when the related receive is started. If an MPI implementation chooses to use internal buffering, the send operation may be already completed before it is enabled, i.e., the receive is started.
• A synchronous mode send operation is enabled exactly when the related receive is started and must not complete before it is enabled.
• A buffered mode send operation is a priori enabled.
• A ready mode send can be started only when it is already enabled, i.e., the related receive is started.
• For a collective broadcast, the operation at a particular MPI process is enabled exactly when all other MPI processes in the group have started their related broadcast operation.