IVE Data Format

All data loaded into IVE must be in netCDF (network Common Data Form) format. The data can be either in a single file containing all desired data, or in multiple files where each file contains the data for single value of a specified dimension and the alphanumeric file name includes the numerical value of the specified dimension (e.g. each file might contain the data for one time in an multi-time data set). The following document describes

Elements of the NetCDF File

A netCDF file consists of four components: dimensions, variables, attributes, and data.

Dimensions

IVE will slice and display data in four-dimensional arrays. The FORTRAN convention for writing such an array is `sample(nx,ny,nz,nt)' where, according to the FORTRAN standard, the first index varies most rapidly. The same array would be written in C as `sample(nt,nz,ny,nx)'. In the remainder of this section we will refer to the order of the array indices using the FORTRAN convention. The first three array indices are assumed to represent variations along three spatial coordinates; the fourth index represents time. It is not necessary that the actual data be four dimensional. For example, if the data depends only on x and z the netCDF array should be declared to be four dimensional, and ny and nt should both be set to 1. Whenever a dimension is set to 1, the slider associated with the corresponding ordinate will not appear in the window/slicer widget (except in the case of a data set in multiple files).

Dimensions are declared in the first section of the netcdf file which begins with the keyword `dimensions' Two examples of the dimensions section of a netcdf file are:

dimensions:
and
dimensions:
In both examples, the time dimension is unlimited, which allows the data to contain an arbitrary number of time levels. Only the last dimension in the data set can be unlimited. Observe that, as illustrated by the second example, the spatial coordinates need not be labeled x, y and z.

Variables

All data arrays contained in the netCDF file must be declared according to the netCDF standard by specifying each array's dimensions and the data type (such as `float' for floating point). Variables appear at the beginning of the netCDF file after the `variables' keyword. An example declaring the floating-point array `sample' is as follows:
variables:
Note that the unlimited dimension appears first and the order in which the dimensions are declared is opposite that used in standard FORTRAN. (The ordering is consistent with standard C.)

Attributes

NetCDF files contain two types of attributes: global attributes, which pertain to the data set as a whole, and per field attributes, which apply only to individual data fields. Those attributes recognized by the default ive transforms are described below. IVE can utilize netCDF files in which no attributes are specified. It is, however, necessary to specify at least some of the following attributes in order to flag missing data, or to indicate staggered mesh locations, or to describe the physical units associated with dimensional data. Additional attributes may appear in netCDF data sets, but these are ignored by the default IVE transforms. Customized IVE transform routines can, however, be written to recognize and use such additional attributes.

Global Attributes

The Global attributes define the size of the physical domain and the units in which to express and to display the physical coordinates. All unit names and conversions between units are defined via the Unidata program udunits. If no units are included in the attributes, IVE treats the data as dimensionless and assumes a spacing of one dimensionless unit between each data point along each coordinate axis. The global attributes recognized by IVE are as follows. In accordance with the netCDF standard, all global attributes begin with a colon.

If the `units' attribute is specified and the `display_units' attribute is omitted, the coordinate will be labeled using the physical units.

Per Field Attributes

The per field attributes define the physical domain over which each particular field is defined, the physical units in which the field is expressed, the units in which the variable is to be displayed and a flag for any missing data. Let `sample' be the name of a data field. All per field attributes begin with the characters `sample:' The global attributes recognized by IVE are as follows.

Data

The data contained in each variable (such as `sample') is specified in the final section of the netCDF file as per the netCDF standard.

Splitting data into multiple files

IVE allows a data set to be spread over several netCDF files. This allows the individual netCDF files to be smaller, and allows some control over how much of the multiple file data set is read into memory during an IVE session.

In a multiple file data set the data is split along a "file coordinate," which is one of the four coordinates x, y, z or t. Each individual netCDF file contains all the data associated with a single value of the "file coordinate." That netCDF file must also contain the numerical value of the "file coordinate" in its alphanumeric name. Depending on the parameters set by the MULTIPLE_READ command, IVE will treat these netCDF files individually or combine selected files into a larger data set in a manner that is transparent to the user.

Examples of IVE compatible netCDF files

Simplest File Construct

The following is the header from a simple netCDF file containing two arrays `A(nx,ny,nz,nt)' and `B(nx,ny,nz,nt)' that will be plotted by IVE using the default transforms.
netcdf simple {

dimensions:
    nx = 50 ;
    ny = 50 ;
    nz = 30 ;
    nt = 10 ;
    nt1 = 1 ;

variables:

    float A(nt,nz,ny,nx);
    float B(nt1,nz,ny,nx);

(Note that the header information in an existing netCDF file may be obtained using the command ncdump -h filename ) No attributes are specified in this netCDF file. IVE uses nondimensional units to specify the domain for the field `A' as 1 < x < 50, 1 < y < 50, 1 < z < 30, and 1 < t < 10. The domain for field `B' is identical to that for `A', except that there is no time dependence. As a consequence, the time-coordinate slider will not appear in the window/slicer widget when field `B' is selected.

File With Explicit Specification of Physical Grid Point Locations

The following portion of a netCDF file contains five variables. A single four-dimensional data array `A(lat,lon,level,time)' contains the data to be plotted. Four one-dimensional arrays, `lat', `lon' `level', and `time' contain the coordinates of the physical grid. These coordinates may be irregularly spaced and may have attributes stating their physical units.

netcdf explicit_grid{

dimensions:
    lon = 5;
    lat = 5;
    level = 5;
    time = 7;

variables:

    float A(time,level,lat,lon);
      A:units = "meters/second";

    float lon(lon);

      lon:units = "degrees_east";

    float lat(lat);

      lat:units = "degrees_north";

    float level(level);

      level:units = "millibars";

    float time(time);

      time:units = "hours";

//global attributes:

    :x_label = "longitude";
    :y_label = "latitude";
    :z_label = "level";
    :t_label = "time";
data:
    lon = -180, -90, 0, 90, 180 ;
    lat = -90, 45, 0, 45, 90 ;
    level = 1000, 850, 700, 500, 300 ;
    time = 0, 2, 4, 6, 8, 10, 12 ;
(The preceding lists both the netCDF header and the values of the "coordinate variables," i.e., variables that are also dimensions. This type of listing may be obtained from an existing netCDF file using the command ncdump -c filename . The double slash // precedes a comment line in the netCDF file.)

IVE sets the the domain limits to match the first and last data point in each coordinate variable. In this example, the sliders in the window/slicer widget will show the limits -180 < lon < 180, -90 < lat < 90, 1000 > lev > 300, and 0 < tim < 12. Note that IVE uses the first three letters in the name of the global label attributes to label the sliders. IVE recognizes `lon',`lat',`level' and `time' as coordinate variables and automatically treats them as if they had the attribute "no_button" set to `1'

File With Implicit Specification of Physical Grid Point Locations

As the number of grid points in a data array increases, it becomes awkward to explicitly list the physical location of each coordinate point in the coordinate variable arrays. If the variables are evenly spaced, this can be avoided as per the following example.
netcdf implicit_grid{

dimensions:
    lon = 101;
    lat = 101;
    level = 5;
    time = UNLIMITED ; //(7 currently)

variables:

    float A(time,level,lat,lon);
      A:units = "meters/second";

    float level(level);

      level:units = "millibars";

    float time(time);

      time:units = "hours";

//global attributes:

    :x_min = -180.f;
    :x_max = 180.f;
    :x_units = "degrees_east";
    :x_label = "longitude";

    :y_min = -90.f;
    :y_max = 90.f;
    :y_units = "degrees_north";
    :y_label = "latitude";

    :z_label = "level";
    :t_label = "time";

data:
    level = 1000, 850, 700, 500, 300 ;
    time = 0, 2, 4, 6, 8, 10, 12 ;

IVE uses the default transform routines to linearly interpolate between the array indices and the physical location along the x and y coordinates. Let `i' be the value of the first array index. Since :x_delta is not specified, `i' is mapped to a position along the x (or longitude) coordinate via the relation 

       :x_min + (i-1)*(:x_max - :x_min)/(lon-1)
The sliders in the window/slicer widget will look like those in the preceding example. The only difference is that the "lon" and "lat" sliders will not be subject to the "gravity" that forces the indicator to stop only at values corresponding to grid points that are actually present in the data set. Gravity is typically useful only when there are a small number of data points along the coordinate axis.

If :x_delta were specified, `i' would be mapped to a position along the x (or longitude) coordinate via the relation 

       :x_min + :x_delta*(i-1)
If :x_delta and A:x_min were both specified, `i' would be mapped to a position along the x (or longitude) coordinate via the relation 
       A:x_min + :x_delta*(i-1)
which allows `A' to be staggered with respect to other fields along the 1st coordinate axis.

Multiple Files

This is an example of Implicit Specifiation with time as the "file coordinate." The data is organized into three netCDF files containing the data at times 1000, 1500 and 2000, respectively. The names of the netCDF files are data_1000.nc, data_1500.nc, and data_2000.nc. Here is the example for data_1500.nc:
netcdf data_1500{

dimensions:

variables:

//global attributes:

data:

The data_1000.nc and data_2000.nc would have "time" set to 1000 and 2000, respectively. If the same data set was used with a "file coordinate" of z, the file names would have been data_1000.nc, data_850.nc, data_700.nc, data_500.nc, and data_300.nc and the data_850.nc file would look as follows:

netcdf data_850{

dimensions:

variables:

//global attributes:

data:

back to index