Associated Variable Input and Output
Binary data stored in files often consists of a repetitive series of arrays or structures. A common example is a series of images or a series of arrays. PV-WAVE associated file variables offer a convenient and efficient way to access data that comprises a sequence of identical arrays or structures.
An associated variable is a variable that maps definitions of an array or structure variable onto file contents. The file is treated as an array of these repeating units of data. The file’s first array or structure has an index of 0, the second has index 1, and so on. The general form for using ASSOC is:
ASSOC(unit, array_definition [, offset])
For examples showing how to use the
offset parameter, refer to a later section,
"Using the Offset Parameter".
Associated variables do not use memory like a normal variable. Instead, when an associated variable is subscripted with the index of the desired array or structure within the file, PV-WAVE performs the I/O operation required to access that entire block of data.
Advantages of Associated File Variables
Associated file variables offer the following advantages over READU and WRITEU for binary I/O. For these reasons, associated variables are the most efficient form of I/O.
I/O occurs when an associated file variable is subscripted. Thus, it is possible to perform I/O within expressions, without a separate statement.
The size of the data set is limited primarily by the maximum size of the file containing the data, instead of the maximum memory available. Data sets too large for memory can be easily accommodated.
You do not have to declare the maximum number of arrays or structures contained in the file.
Associated variables simplify access to the data. Direct access to any element in the file is rapid and simple—there is no need to calculate offsets into the file and/or position a file pointer prior to an I/O operation.
Working with Associated File Variables
Assume that a file named today.dat exists, and that this file contains a series of 10-by-20 arrays of floating-point data. The following two statements open the file and create an associated file variable mapped to the file:
; Open the file.
OPENU, 1, 'today.dat'
; Define an associated file variable. Using the Nozero keyword
; with FLTARR increases efficiency since ASSOC ignores the value
; of the resultant array, anyway.
A = ASSOC(1, FLTARR(10, 20, /Nozero))
note | The order of these two statements is not important—it would be equally valid to call ASSOC first and then open the file. This is because the association is between the variable and the logical file unit, not the file itself. |
You may opt to close the file, open a new file using the same LUN, and then use the associated variable without first executing a new ASSOC. Naturally, an error occurs if the file is not open when the file variable is subscripted in an expression, or if the file is open for the wrong type of access (for example, trying to assign to an associated file variable with a file opened with OPENR for read-only access).
As a result of executing the two statements above, the variable A is now an associated file variable. Executing the statement:
INFO, A
produces the following response:
A FLOAT = File<today.dat> Array(10, 20)
The associated variable A maps the definition of a 10-by-20 floating-point array onto the contents of the file today.dat. Thus, the response from the INFO procedure shows it to be a two-dimensional floating-point array.
note | Only the array’s form is used by ASSOC. The value of the expression is ignored. |
The ASSOC command doesn't require that you use a particular combination of dimensions to index into a file, although you may have reasons to prefer one combination of dimensions over another. For example, assume a number of 128-by-128 byte images are contained in a file. The command:
row = ASSOC(1, BYTARR(128))
maps the file into rows of 128 bytes each. Thus, row(3) is the fourth row of the first image, and row(128) is the first row of the second image. On the other hand, the command:
image = ASSOC(1, BYTARR(128,128))
maps the file into entire images. Now, image(4) is all 16384 values of the fifth image.
How Data is Transferred into Associated Variables
Once a variable has been associated with a file, data is read from the file whenever the associated variable appears in an expression with a subscript. The position of the array or structure read from the file is given by the value of the subscript. The following statements give some examples of using file variables:
; Copy the contents of the first array into the normal variable
; Z. Z is now a 10-by-20 floating-point array.
Z = A(0)
; Compute the sum of the first 10 arrays (Z was initialized in
; the previous statement to the value of the first array. This
; statement adds the following nine to it.).
FOR I = 1,9 DO Z = Z + A(I)
; Read the fourth array and plot it.
PLOT, A(3)
; Subtract array 4 from array 5, and plot the result. The result
; of the subtraction is not saved after the plot is displayed.
PLOT, A(5) - A(4)
An associated file variable only performs I/O to the file when it is subscripted. Thus, the following two statements do not cause I/O to happen:
; This assignment does not transfer data from file to variable B
; because A is not subscripted. Instead, B becomes an associated
; file variable with the same dimensions, and to the same LUN,
; as A.
B = A
; This does not result in the value 23 being transferred to the
; file because variable B (which became an associated file vari
; able in the previous statement) is not subscripted. Instead,
; B becomes a scalar integer variable containing the value 23.
; It is no longer an associated file variable.
B = 23
Subscripting Associated File Variables During Input
When the associated file variable is defined to be an array, it is possible to subscript into the array being accessed during input operations. For example, for the variable A defined above:
; Assigns the value of the first floating-point element of the
; second array within the file to the variable Z. The rightmost
; subscript is taken as the index into the file causing PV-WAVE
; to read the entire second array into memory. This resulting
; array expression is then further subscripted by the remaining
; subscripts.
Z = A(0,0,1)
note | Although this ability can be convenient, it can also be very slow because every access to an individual array element causes the entire array to be read from disk. Unless only one element of the array is desired, it is much faster to assign the contents of the array to another variable by subscripting the file variable with a single subscript, and then access the individual array elements from the variable. |
Efficiency in Accessing Arrays
To increase the efficiency of reading arrays, make their length an integer multiple of the physical block size of the disk holding the file. Common values are 512, 1024, and 2048 bytes. For example, on a disk with 512-byte blocks, one benchmark program required approximately one-eighth of the time required to read a 512-by-512 byte image that started and ended on a block boundary, as compared to a similar program that read an image that was not stored on even block boundaries.
Using the Offset Parameter
The offset parameter to ASSOC specifies the position in the file at which the first array starts. It is useful when a file contains a header followed by data records.
Specifying Offsets Under UNIX and Windows
The offset is given in bytes. For example, if a file uses the first 1024 bytes of the file to contain header information, followed by 512-by-512 byte images, the statement:
image = ASSOC(1, BYTARR(512, 512), 1024)
skips the header by providing a 1024 byte offset before any image data is read.
Writing Associated Variable Data
When a subscripted associated variable appears on the left side of an assignment statement, the expression on the right side is written into the file at the given array position. For example:
; Zeroes sixth record. By default, every value in a newly created
; floating-point array is set equal to zero, unless Nozero is set.
A(5) = FLTARR(10,20)
; Writes ARR into sixth record after any necessary conversions.
A(5) = ARR
; Averages records J and J+1 and writes the result into record J.
A(J) = (A(J) + A(J+1))/2
note | When writing data, only a single subscript (specifying the index of the affected array or structure in the file) is allowed. Thus, it is not possible to index individual elements of associated arrays during output, although it is allowed during input. To update individual elements of an array within a file, assign the contents of that array to a normal array variable, modify the copy, and write the array back by assigning it to the subscripted associated variable. |
Binary Data from UNIX FORTRAN Programs
Binary data files generated by FORTRAN programs under UNIX contain an extra longword before and after each logical record in the file. ASSOC does not interpret these extra bytes, but considers them to be part of the data. Therefore, do not use ASSOC to read such files; use READU and WRITEU instead. You can find an example of using PV-WAVE to read data generated by FORTRAN programs under UNIX in
"Reading UNIX FORTRAN-Generated Binary Data".
Version 2017.1
Copyright © 2019, Rogue Wave Software, Inc. All Rights Reserved.