Re: data access functions

Samuel A. Fineberg (fineberg@nas.nasa.gov)
Wed, 12 Mar 1997 14:48:01 -0800

In message John May writes
>Rajeev Thakur writes:
>> Removing etypes and restricting accesses to whole filetypes would make
>> implementations easier. For many common access patterns, such a change
>> should not cause undue hardship on the application programmer. But
>> for some access patterns, it may. Consider the foll. filetype
>>
>> |--------------------|.............|-----------------|
>> contiguous hole contiguous
>>
>> If there isn't enough memory to read this entire filetype at once,
>> then reading it in parts would require some care in setting and
>> changing the view.
>
>I agree, and the argument also works for writes: you might
>want to build up an elaborate filetype partition of the file,
>but only fill up the partition incrementally.
>
>In general, I think we need to be very conservative about
>making changes at this point. Most of the functionality
>in the chapter has been implemented by one or more of us,
>and we've learned a lot about what's feasible and what isn't
>through these implementations. Many things that looked
>straightforward turned out to have subtle problems. Rash
>changes made now will be hard to fix. I'm already nervous
>about filters, even though they look like a good idea.
>

I think the real issue is whether or not we want to support the writing
of partial filetypes. I don't see any reason why sane programs would
want to read and write partial filetypes. The original reason for
reading/writing partial filetypes was to use filetypes to emulate Vesta
file views, i.e., to control file striping. Using filetypes in this way
may improve performance on some systems, but the performance improvement
will be totally non-portable.

The example above is another reason for reading partial filetypes, however,
I do not think that this case is likely. If the system you are using
really has too little memory to load entire filetypes then the filetype
should be redefined or a simple "least common denominator" filetype should
be used.

I.e., why not use the following filetype:

|--------------------|.............|
contiguous hole

Or, if you don't have enough memory to read a "contiguous" region, why not
just use:

|---------|
contiguous

There is no real benefit to having holes in filetypes if you aren't
accessing filetypes that span the hole.

Now, if we can agree to make the change to prohibit accessing partial
filetypes, eliminating the etype isn't really a change. There were only
three reasons why we initially created the etype:

1. There needed to be a common type signature between the buftype and the
filetype

- this is now just explicitly mandated.

2. We needed a data size independent way to specify absolute file offsets
(i.e., we didn't want to specify offsets in bytes)

- this need was eliminated a long time ago when we eliminated
absolute offsets

3. We needed a way to express offsets in to the middle of a filetype

- eliminated by this change

So, if we do decide to make this change, then eliminating etypes is a
necessity because they will no longer serve any purpose.

Sam
-----
Samuel A. Fineberg Address: Numerical Aerodynamic Simulation
MRJ Technology Solutions NASA Ames Research Center M/S 258-6
Supercomputer Applications Segment Moffett Field, CA 94035-1000
e-mail: fineberg@nas.nasa.gov Phone: (415)604-4319(voice) (415)966-8669(FAX)
See my home page at URL: "http://www.dcs.uky.edu/~fineberg/"