Re: Concern With MPI IO Type Constructor Limitation

Bill Nitzberg (nitzberg@nas.nasa.gov)
Mon, 19 Aug 1996 10:26:47 -0700

In message <199608191506.LAA03677@cpu91772.pwfl.com> Wright_Dave writes
> I was reviewing the MPI-IO API, especially the part of describing the
> type of data in a file. It appears that the non-header portion of the
> file must be all one type, for example float/real, for MPI-IO to
> operate on the file.
>
> Is this true?
>
> If so, how do we support files that are a mix of integers and floats?
> Some unstructured codes, for example, store index information along
> with floating point numbers on each "record" in the file. How do we
> use MPI-IO in this case to efficiently read/write the file? I don't
> think the we should have to redefine the file layout each time a
> different type of information needs to be read or written.
>
> Please comment.
>
> ----------------------------------------------------------------------
> Dave Wright High Performance Computing/Computational Fluid Dynamics
> voice:407-796-8477 fax:407-796-6049 internet: wrightdv@pwfl.com
> Pratt & Whitney, PO Box 109600, M/S 712-63, West Palm Beach, FL 33410

No, this is not true.

MPI I/O directly supports reading and writing files which contain
data of any combination of types in at least three different ways:
the etype, the filetype, and by changing the file view after the
file is opened.

Both the etype and filetype are completely general MPI datatypes
and can be structured types (such as (index, value) pairs) in addition
to the simple types like integers and floats. Even more general
file data is supported by changing the file view (via the
old MPI_File_control or new MPI_View routines).

For example, a file with a 100 character header followed by some
number of index/value pairs:

header etype-1 etype-2

| | | | |
char [100], int, int, int, double, int, int, int, double, ...
| | | | |

"run 27A" x y z temp x y z temp

This would be defined in MPI I/O by the following (pardon the
pseudo-syntax):

/*
* Create an etype to represent an (x,y,z,temp)
*/
MPI_Datatype etype;
int blocklengths[] = {1, 1};
MPI_Aint displacements[] = {0, 3*sizeof(int)};
MPI_Datatype types[2];

MPI_Type_contiguous(3, MPI_INT, &types[0]);
types[1] = MPI_DOUBLE;

MPI_Type_struct(1, blocklengths, displacements, types, &etype);

/*
* Create filetype which partitions etypes onto processes
*/
MPI_Datatype filetype;
...

/*
* Open the file
*/
MPI_Open(comm, "file", MPI_RDONLY, info, &fh);
MPI_View(fh, 100*sizeof(char), etype, filetype, info);

/*
* Read the first (x,y,z,temp)
*/
MPI_Status status;
struct point {
int x, y, z;
double temp;
} point;

MPI_Read_next(fh, &point, etype, 1, &status);
...