I wrote up this example for your general comments to see if we
agree that they are worth pursuing. The optimal data structures for
the calling arguments haven't been completely worked out, but
this is the general idea. (Fortran might be a problem).
If we agree that they are worth pursuing and designing properly,
I will finish them up by Monday. There are many algorithms that
operate over sub-blocks. I need to check some of my literature.
Here is a primitive discussion and example. Feedback is welcomed!!
Non-Uniform Sub-blocks Discussion
-----------------------------------
These new collective functions accomodate send and receive buffers that
are represented by matrix sub-blocks and strides that represents a
sub-block as a position in the matrix. The position is
indicated by the displacement parameter.
There are several functions that could be included.
1.A Scatter function similar to MPI-SCATTERV with an array of
sendtypes, and an array of displacement types.
2.A Gather function similar to MPI_GATHERV with an array of receivetypes
and an array of displacement types.
These are in the spirit of Lloyd's proposal in 5.10 which deals with
arrays of data types for send and receive buffers. The buffer datatypes
could be sub-block matrices. The strides would also be arrays
of datatypes, and these could be ranges of rows and columns.
This may be used in collective operations on submatrices,
on image sub-blocks or for kronecker product operations.
In the worst case, the blocks all differ in ROWS and/or COLUMNS.
--------------------------------------------------------------------
--------------------------------------------------------------------
Example 1: Given the matrix with the sub-blocks
|2x4 4x2|
|2x4 |
------------------------------------------------------------------
------------------------------------------------------------------
An example of how GATHER,followed by SCATTER can be used with
a scalar and a non-scalar operation on blocks of matrices.
Note that the user could define strides to indicate sub-block
positions in a matrix.
NOTE:
Send buffers are not NULL addresses for the root in a Gather and
this distinguishes the operation from MPI_Gather. Conversely, receive
buffers are not NULL addresses and this distinguishes the Scatter
operation from MPI_SCATTER.
Gather
Root send buffer (sends to itself)
2x4
Process send buffer Root receive buffer
4x2 2x4 4x2 2x4
2x4
If Operate = scalar operation then
Scatter
Root Send Buffer Root Receive buffer
2x4 4x2 2x4 2x4
Process receive buffer
4x2
2x4
else
if Operate = non-scalar = (2x2 left kronecker product) then
Scatter
Root Send Buffer Root Receive Buffer
4x8 8x4 4x8 4x8
Process receive buffer
8x4
4x8
--------------------------------------------------------------
Strides:
If strides are ranges of rows and columns, then strides
represent blocks of a matrix.
A stride representing a 2x4 matrix occupying the upper left
hand corner of a matrix would have a row range of 1-2 and a
column range of 1-4.
Growing in dimension:
In the case of Kronecker product operations, the matrix blocks are
growing in size. The stride also grows.