Re: Canonical (External?) Data Representation

Albert Cheng (acheng@ncsa.uiuc.edu)
Tue, 18 Feb 1997 17:51:03 -0600

At 10:35 AM 2/17/97 +0000, James Cownie wrote:
>Albert points out the desire to be able to read files written with
>MPI/IO on sequential machines, or read files written on sequential
>machines via MPI-IO.
>
>Unfortunately even the canonical representation cannot make any
>promises about either of these operations.
>
>The canonical representation allows a file written by MPI-IO on one
>machine to be read by MPI-IO on another machine. It does not make any
>statements about reading such a file from a non MPI-IO environment.

Jim, you are correct that the paragraph of the MPI_DATA_CANONICAL
access mode does not specify compatibility with non-MPI environments.
But in view of other context of the IO chapter, it can be concluded
that way. Let me explain.

First of all, the canonical representation defines the bytes size
and bit pattern of each data value. So, for each MPI-view, there
can be only one bit pattern of the data file if one walks through
it byte by byte. (It is not as absolute as I present because the
bytes under the displacement can be any random values and LOGICAL
value TRUE can be any non-zero patterns. But you know what I
mean. :-)

Secondly, it is expected the MPI implementation would provide the
mechanism/tools for conversion between files in and outside the
MPI-environment with the requirement of maintaining correspondance
between byte offsets (p.249, line 38-42). Therefore, for each
MPI-view, there can be only one bit pattern of the data file outside
of the MPI-environment, implying a "unique outside" data file.

With this defined uniqueness, a non-MPI-IO environment has a defined
way to interpret the data file for all MPI-environments.

>The reason for this omission is that there are many native file
>formats (particularly for files written from Fortran), and therefore
>MPI cannot be both canonical and inter-operable.

True. The canonical data representation cannot solve those problems
but it gives the users a defined means to convert the native files
into the canonical-formatted files which would work for all
MPI-environments. I would expect a high-qualify vendor would provid
tools for conversion in and out of their own MPI-environment but if
a user has to move data files between different platforms, the
canonical formatted file is a means they can rely on.

--
Albert