Re: Canonical (External?) Data Representation

Leslie Hart (hart@nipmuc.fsl.noaa.gov)
Mon, 17 Feb 1997 10:31:25 -0700 (MST)

The proposed standard does provide for the equivalents of copy and move
functions to be present. The exact name and syntax are not provided, but
they are required. There is also verbage regarding import/export of files
from/to the "native" file system to/from the "MPI" system (no-one precludes this
from being cp/mv/ln -s in Un*x). "High-quality" implementations are expected to
provide retention of seek locations when a file is imported/exported. This
still does not solve the record oriented language/system problem, but does
solve a large number of problems that the user may face.

In short, the Canonical (or External) representation(s) together with import/
export (and in a high-quality implementaion, retention of seek addresses) gives
the user a strong clue as to what is in a sequential file. Hopefully the user
will only have to resort to "read on one process using MPI-IO, and write using
the normal mechanism" on a very small number of "low quality" implementations.

Leslie Hart (hart@fsl.noaa.gov)
>
> Albert points out the desire to be able to read files written with
> MPI/IO on sequential machines, or read files written on sequential
> machines via MPI-IO.
>
> Unfortunately even the canonical representation cannot make any
> promises about either of these operations.
>
> The canonical representation allows a file written by MPI-IO on one
> machine to be read by MPI-IO on another machine. It does not make any
> statements about reading such a file from a non MPI-IO environment.
>
> The reason for this omission is that there are many native file
> formats (particularly for files written from Fortran), and therefore
> MPI cannot be both canonical and inter-operable.
>
> The simplest solution which is guaranteed to work is very likely going
> to remain "read on one process using MPI-IO, and write using the
> normal mechanism".
>
> While this is a depressing conclusion, please note that this is *not*
> an MPI problem, but an existing problem that MPI happens to have
> bumped into.
>
> -- Jim
>
> James Cownie
> Dolphin Interconnect Solutions
> Phone : +44 117 9071438
> E-Mail: jcownie@dolphinics.com
>