HDF User’s Guide
Version 4.2.5[Top] [Prev][Next]
|
|
HDF User’s GuideVersion 4.2.5 |
[Top] [Prev][Next] |
Chapter 3 -- Scientific Data Sets (SD API)
3.1 Chapter Overview
This chapter describes the scientific data model and the interface routines provided by HDF for creating and accessing the data structures included in the model. This interface is known as the SD interface or the SD API.
3.2 The Scientific Data Set Data Model
The scientific data set, or SDS, is a group of data structures used to store and describe multidimensional arrays of scientific data. Refer to Figure 3a for a graphical overview of the SD data set. Note that in this chapter the terms SDS, SD data set, and data set are used interchangeably; the terms SDS array and array are also used interchangeably.
A scientific data set consists of required and optional components, which will be discussed in the following subsections.
FIGURE 3a The Contents of a Scientific Data Set3.2.1 Required SDS Components
Every SDS must contain the following components: an SDS array, a name, a data type, and the dimensions of the SDS, which are actually the dimensions of the SDS array.
SDS ArrayAn SDS array is a multidimensional data structure that serves as the core structure of an SDS. This is the primary data component of the SDS model and can be compressed (refer to Section 3.5.2 on page 47 for a description of SDS compression) and/or stored in external files (refer the Section 3.5.3.3 on page 54 for a description of external SDS storage). Users of netCDF should note that SDS arrays are conceptually equivalent to variables in the netCDF data model1.
An SDS has an index and a reference number associated with it. The index is a non-negative integer that describes the relative position of the data set in the file. A valid index ranges from 0 to the total number of data sets in the file minus 1. The reference number is a unique positive integer assigned to the data set by the SD interface when the data set is created. Various SD interface routines can be used to obtain an SDS index or reference number depending on the available information about the SDS. The index can also be determined if the sequence in which the data sets are created in the file is known.
In the SD interface, an SDS identifier uniquely identifies a data set within the file. The identifier is created by the SD interface access routines when a new SDS is created or an existing one is selected. The identifier is then used by other SD interface routines to access the SDS until the access to this SDS is terminated. For an existing data set, the index of the data set can be used to obtain the identifier. Refer to Section 3.4.1 on page 27 for a description of the SD interface routine that creates SDSs and assigns identifiers to them.
SDS NameThe name of an SDS can be provided by the calling program, or is set to "DataSet" by the HDF library at the creation of the SDS. The name consists of case-sensitive alphanumeric characters, is assigned only when the data set is created, and cannot be changed. SDS names do not have to be unique within a file, but their uniqueness makes it easy to semantically distinguish among data sets in the file.
Data TypeThe data contained in an SDS array has a data type associated with it. The standard data types supported by the SD interface include 32- and 64-bit floating-point numbers, 8-, 16- and 32-bit signed integers, 8-, 16- and 32-bit unsigned integers, and 8-bit characters. The SD interface also allows the creation of SD data sets consisting of data elements of non-standard lengths (1 to 32 bits). See Section 3.7.11 on page 75 for more information.
DimensionsSDS dimensions specify the shape and size of an SDS array. The number of dimensions of an array is referred to as the rank of the array. Each dimension has an index and an identifier assigned to it. A dimension also has a size and may have a name associated with it.
A dimension identifier is a positive number uniquely assigned to the dimension by the library. This dimension identifier can be retrieved via an SD interface routine. Refer to Section 3.8.1 on page 77 for a description of how to obtain dimension identifiers.
A dimension index is a non-negative number that describes the ordinal location of a dimension among others in a data set. In other words, when an SDS dimension is created, an index number is associated with it and is one greater than the index associated with the last created dimension that belongs to the same data set. The dimension index is convenient in a sequential search or when the position of the dimension among other dimensions in the SDS is known.
The size of a dimension is a positive integer. Also, one dimension of an SDS array can be assigned the predefined size
SD_UNLIMITED(or0). This dimension is referred to as an unlimited dimension, which, as the name suggests, can grow to any length. Refer to Section 3.5.1.3 on page 41 for more information on unlimited dimensions.Names can optionally be assigned to dimensions, however, dimension names are not treated in the same way as SDS array names. For example, if a name assigned to a dimension was previously assigned to another dimension the SD interface treats both dimensions as the same data component and any changes made to one will be reflected in the other.
Important Note:
HDF4 allows a dimension and a one-dimensional SDS to be given the same name. Prior to HDF4.2r2, however, the library did not always adequately track object types; when a dimension and a one-dimensional SDS shared a name, writing to the SDS or the dimension could cause data corruption to the other. The corrupted data was unrecoverable.
This problem was fixed in Release 4.2r2 and such data corruption will not occur in files created with a 4.2r2 or later library. Note, however, that the fix is effective only in new files; a dimension and a one-dimensional SDS of the same name that were created with a pre-4.2r2 HDF4 Library remain vulnerable to data corruption if an application is unaware of the potential conflict. To safely handle pre-4.2r2 files, the library now provides two functions, SDgetnumvars_byname and SDnametoindices. SDgetnumvars_byname can be used to determine whether a name is unique. If the function reports one ('1') variable by that name, the name is unique and no further precaution needs to be taken. If the name is not unique, i.e., the number of variables by that name is greater than one, SDnametoindices must then be used to retrieve the index and the type of each variable with that name. The desired variable can then can be safely selected via its index. These functions are described in detail in this User's Guide and the HDF4 Reference Manual.
A similar problem is possible when a multi-dimensional SDS and a dimension are created with the same name by a pre-4.2r2 library. The HDF Group has not seen such a failure, however, and it is thought to be very unlikely. Note that the fix introduced in Release 4.2r2 also prevents data corruption from happening for this situation even though the data was created with libraries prior to 4.2r2, assuming no corruption had yet occurred.
3.2.2 Optional SDS Components
There are three types of optional SDS components: user-defined attributes, predefined attributes, and dimension scales. These optional components are only created when specifically requested by the calling program.
Attributes describe the nature and/or the intended usage of the file, data set, or dimension they are attached to. Attributes have a name and value which contains one or more data entries of the same data type. Thus, in addition to name and value, the data type and number of values are specified when the attribute is created.
User-Defined AttributesUser-defined attributes are defined by the calling program and contain auxiliary information about a file, SDS array, or dimension. They are more fully described in Section 3.9 on page 90.
Predefined AttributesPredefined attributes have reserved names and, in some cases, predefined data types and/or number of data entries. Predefined attributes are useful because they establish conventions that applications can depend on. They are further described in Section 3.10 on page 101.
Dimension ScalesA dimension scale is a sequence of numbers placed along a dimension to demarcate intervals along it. Dimension scales are described in Section 3.8.4 on page 79.
3.2.3 Annotations and the SD Data Model
In the past, annotations were supported in the SD interface to allow the HDF user to attach descriptive information (called metadata) to a data set. With the expansion of the SD interface to include user-defined attributes, the use of annotations to describe metadata should be eliminated. Metadata once stored as an annotation is now more conveniently stored as an attribute. However, to ensure backward compatibility with scientific data sets and applications relying on annotations, the AN annotation interface, described in Chapter 10, Annotations (AN API) can be used to annotate SDSs.
There is no cross-compatibility between attributes and annotations; creating one does not automatically create the other.
3.3 The SD Interface
The SD interface provides routines that store, retrieve, and manipulate scientific data using the SDS data model. The SD interface supports simultaneous access to more than one SDS in more than one HDF file. In addition, the SD interface is designed to support a general scientific data model which is very similar to the netCDF data model developed by the Unidata Program Center2.
For those users who have been using the DFSD interface, the SD interface provide a model compatible with that supported by the DFSD interface. It is recommended that DFSD users apply the SD model and interface to their applications since the DFSD interface is less flexible and less powerful than the SD interface and will eventually be removed from the HDF library.
This section specifies the header file to be used with the SD interface and lists all available SD interface routines, each of which is accompanied by its purpose and the section where the routine is discussed.
3.3.1 Header and Include Files Used by the SD Interface
The "mfhdf.h" header file must be included in programs that invoke SD interface routines. FORTRAN-77 users should refer to Section 2.5.3 on page 16.
3.3.2 SD Interface Routines
All C routines in the SD interface begin with the prefix "SD". The equivalent FORTRAN-77 routines use the prefix "sf". These routines are categorized as follows:
- Access routines initialize and terminate access to HDF files and data sets.
- Read and write routines read and write data sets.
- General inquiry routines return information about the location, contents, and description of the scientific data sets in an HDF file.
- Dimension routines access and define characteristics of dimensions within a data set.
- Dimension scale routines define and access dimension scales within a data set.
- User-defined attribute routines describe and access characteristics of an HDF file, data set or dimension defined by the HDF user.
- Predefined attribute routines access previously-defined characteristics of an HDF file, data set, or dimension.
- Compression routines compress SDS data.
- Chunking/tiling routines manage chunked data sets.
- Miscellaneous routines provide other operations such as external file, n-bit data set, and compatibility operations.
The SD routines are listed in the following table and are discussed in the following sections of this document.
TABLE 3A SD Interface Routines
3.3.3 Tags in the SD Interface
A complete list of SDS tags and their descriptions appears in Table D in Appendix A. Refer to Section 2.2.2.1 on page 8 for a description of tags.
3.4 Programming Model for the SD Interface
This section describes the routines used to initialize the SD interface, create a new SDS or access an existing one, terminate access to that SDS, and shut down the SD interface. Writing to existing scientific data sets will be described in Section 3.5 on page 31.
To support multifile access, the SD interface relies on the calling program to initiate and terminate access to files and data sets. The SD programming model for creating and accessing an SDS in an HDF file is as follows:
To access a single SDS in an HDF file, the calling program must contain the following calls:
C: sd_id = SDstart(filename, access_mode);
sds_id = SDcreate(sd_id, sds_name, data_type, rank, dim_sizes);
OR sds_id = SDselect(sd_id, sds_index);
<Optional operations>
status = SDendaccess(sds_id);
status = SDend(sd_id);
FORTRAN: sd_id = sfstart(filename, access_mode)
sds_id = sfcreate(sd_id, sds_name, data_type, rank, dim_sizes)
OR sds_id = sfselect(sd_id, sds_index)
<Optional operations>
status = sfendacc(sds_id)
status = sfend(sd_id)
To access several files at the same time, a program must obtain a separate SD file identifier (sd_id) for each file to be opened. Likewise, to access more than one SDS, a calling program must obtain a separate SDS identifier (sds_id) for each SDS. For example, to open two SDSs stored in two files a program would execute the following series of function calls.
C: sd_id_1 = SDstart(filename_1, access_mode);
sds_id_1 = SDselect(sd_id_1, sds_index_1);
sd_id_2 = SDstart(filename_2, access_mode);
sds_id_2 = SDselect(sd_id_2, sds_index_2);
<Optional operations>
status = SDendaccess(sds_id_1);
status = SDend(sd_id_1);
status = SDendaccess(sds_id_2);
status = SDend(sd_id_2);
FORTRAN: sd_id_1 = sfstart(filename_1, access_mode)
sds_id_1 = sfselect(sd_id_1, sds_index_1)
sd_id_2 = sfstart(filename_2, access_mode)
sds_id_2 = sfselect(sd_id_2, sds_index_2)
<Optional operations>
status = sfendacc(sds_id_1)
status = sfend(sd_id_1)
status = sfendacc(sds_id_2)
status = sfend(sd_id_2)
3.4.1 Establishing Access to Files and Data Sets: SDstart, SDcreate, and SDselect
In the SD interface, SDstart is used to open files rather than Hopen. SDstart takes two arguments, filename and access_mode, and returns the SD interface identifier, sd_id. Note that the SD interface identifier, sd_id, is not interchangeable with the file identifier, file_id, created by Hopen and used in other HDF APIs.
The argument filename is the name of an HDF or netCDF file.
The argument access_mode specifies the type of access required for operations on the file. All the valid values for access_mode are listed in Table 3B. If the file does not exist, specifying
DFACC_READorDFACC_WRITEwill cause SDstart to return aFAIL(or-1) . SpecifyingDFACC_CREATEcreates a new file with read and write access. IfDFACC_CREATEis specified and the file already exists, the contents of this file will be replaced.TABLE 3B File Access Code Flags
The SD interface identifiers can be obtained and discarded in any order and all SD interface identifiers must be individually discarded, by SDend, before the termination of the calling program.
Although it is possible to open a file more than once, it is recommended that the appropriate access mode be specified and SDstart called only once per file. Repeatedly calling SDstart on the same file and with different access modes may cause unexpected results. Note that it has been reported that opening/closing file in loops is very slow; thus, it is not recommended to perform such operations too many times, particularly, when data is being added to the file between opening/closing.
Prior to HDF 4.2r2, the maximum number of open files was limited to 32; but, it now can be up to what the system allowed.
SDstart returns an SD identifier or a value of
FAIL(or-1). The parameters of SDstart are defined in Table 3C on page 29.SDcreate defines a new SDS using the arguments sd_id, sds_name, data_type, rank, and dim_sizes and returns the data set identifier, sds_id.
The parameter sds_name is a character string containing the name to be assigned to the SDS. The SD interface will generate a default name, "
Data Set", for the SDS, if one is not provided, i.e., when the parameter sds_name is set toNULLin C, or an empty string in FORTRAN-77. The maximum length of an SDS name is no longer limited to 64 characters, starting in HDF 4.2r2. Note that when an older version of the library reads a data set, which was created by a library of version 4.2r2 or later and has the name that is longer than 64 characters, the retrieved name will contain some garbage after 64 characters.The parameter data_type is a defined name, prefaced by
DFNT, and specifies the type of the data to be stored in the data set. The header file "hntdefs.h" contains the definitions of all valid data types, which are described in Chapter 2, HDF Fundamentals, and listed in Table 2F on page 14.The parameter rank is a positive integer specifying the number of dimensions of the SDS array. The maximum rank of an SDS array is defined by
H4_MAX_VAR_DIMS(or32), which is defined in the header file "netcdf.h". Note that, in order for HDF4 and NetCDF models to work together, HDF allows SDS to have rank 0. However, there is no intention for data to be written to this type of SDS, but only to store attribute as part of the data description. Consequently, setting compression and setting chunk are disallowed.Each element of the one-dimensional array dim_sizes specifies the length of the corresponding dimension of the SDS array. The size of dim_sizes must be the value of the parameter rank. To create a data set with an unlimited dimension, assign the value of
SD_UNLIMITED(or0) to dim_sizes[0] in C, and to dim_sizes(rank) in FORTRAN-77. See the notes regarding the potential performance impact of unlimited dimension data sets in Section 14.4.3, "Unlimited Dimension Data Sets (SDSs and Vdatas) and Performance" on page 435.Once an SDS is created, you cannot change its name, data type, size, or shape. However, it is possible to modify the data set's data or to create an empty data set and later add values. To add data or modify an existing data set, use SDselect to get the data set identifier instead of SDcreate.
Note that the SD interface retains no definitions about the size, contents, or rank of an SDS from one SDS to the next, or from one file to the next.
SDselect initiates access to an existing data set. The routine takes two arguments: sd_id and sds_index and returns the SDS identifier sds_id. The argument sd_id is the SD interface identifier returned by SDstart, and sds_index is the position of the data set in the file. The argument sds_index is zero-based, meaning that the index of first SDS in the file is 0.
Similar to SD interface identifiers, SDS identifiers can be obtained and discarded in any order as long as they are discarded properly. Each SDS identifier must be individually disposed of, by SDendaccess, before the disposal of the identifier of the interface in which the SDS is opened.
SDcreate and SDselect each returns an SDS identifier or a value of
FAIL(or-1). The parameters of SDstart, SDcreate, and SDselect are further described in Table 3C.3.4.2 Terminating Access to Files and Data Sets: SDendaccess and SDend
SDendaccess terminates access to the data set and disposes of the data set identifier sds_id. The calling program must make one SDendaccess call for every SDselect or SDcreate call made during its execution. Failing to call SDendaccess for each call to SDselect or SDcreate may result in a loss of data.
SDend terminates access to the file and the SD interface and disposes of the file identifier sd_id. The calling program must make one SDend call for every SDstart call made during its execution. Failing to call SDend for each SDstart may result in a loss of data.
SDendaccess and SDend each returns either a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDendaccess and SDend are further described in Table 3C.TABLE 3C SDstart, SDcreate, SDselect, SDendaccess, and SDend Parameter ListsEXAMPLE 1. Creating an HDF file and an Empty SDS.This example illustrates the use of SDstart/sfstart, SDcreate/sfcreate, SDendaccess/sfendacc, and SDend/sfend to create the HDF file named SDS.hdf, and an empty data set with the name SDStemplate in the file.
Note that the Fortran program uses a transformed array to reflect the difference between C and Fortran internal data storages. When the actual data is written to the data set, SDS.hdf will contain the same data regardless of the language being used.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" #define SDS_NAME "SDStemplate" #define X_LENGTH 5 #define Y_LENGTH 16 #define RANK 2 /* Number of dimensions of the SDS */ main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id; /* SD interface and data set identifiers */ int32 dim_sizes[2]; /* sizes of the SDS dimensions */ intn status; /* status returned by some routines; has value SUCCEED or FAIL */ /********************* End of variable declaration ***********************/ /* * Create the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_CREATE); /* * Define the dimensions of the array to be created. */ dim_sizes[0] = Y_LENGTH; dim_sizes[1] = X_LENGTH; /* * Create the data set with the name defined in SDS_NAME. Note that * DFNT_INT32 indicates that the SDS data is of type int32. Refer to * Table 2E for definitions of other types. */ sds_id = SDcreate (sd_id, SDS_NAME, DFNT_INT32, RANK, dim_sizes); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program create_SDS implicit none C C Parameter declaration. C character*7 FILE_NAME character*11 SDS_NAME integer X_LENGTH, Y_LENGTH, RANK parameter (FILE_NAME = 'SDS.hdf', + SDS_NAME = 'SDStemplate', + X_LENGTH = 5, + Y_LENGTH = 16, + RANK = 2) integer DFACC_CREATE, DFNT_INT32 parameter (DFACC_CREATE = 4, + DFNT_INT32 = 24) C C Function declaration. C integer sfstart, sfcreate, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, dim_sizes(2) integer status C C**** End of variable declaration ************************************ C C C Create the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_CREATE) C C Define dimensions of the array to be created. C dim_sizes(1) = X_LENGTH dim_sizes(2) = Y_LENGTH C C Create the array with the name defined in SDS_NAME. C Note that DFNT_INT32 indicates that the SDS data is of type C integer. Refer to Tables 2E and 2I for the definition of other types. C sds_id = sfcreate(sd_id, SDS_NAME, DFNT_INT32, RANK, . dim_sizes) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.5 Writing Data to an SDS
An SDS can be written partially or entirely. Partial writing includes writing to a contiguous region of the SDS and writing to selected locations in the SDS according to patterns defined by the user. This section describes the routine SDwritedata and how it can write data to part of an SDS or to an entire SDS. The section also illustrates the concepts of compressing SDSs and using external files to store scientific data.
3.5.1 Writing Data to an SDS Array: SDwritedata
SDwritedata can completely or partially fill an SDS array or append data along the dimension that is defined to be of unlimited length (see Section 3.5.1.3 on page 41 for a discussion of unlimited-length dimensions). It can also skip a specified number of SDS array elements between write operations along each dimension.
To write to an existing SDS, the calling program must contain the following sequence of routine calls:
To write to a new SDS, simply replace the call SDselect with the call SDcreate, which is described in Section 3.4.1 on page 27.
SDwritedata takes five arguments: sds_id, start, stride, edges, and data. The argument sds_id is the data set identifier returned by SDcreate or SDselect.
Before proceeding with the description of the remaining arguments, an explanation of the term hyperslab (or slab, as it will be used in this chapter) is in order. A slab is a group of SDS array elements that are stored in consecutive locations. It can be of any size and dimensionality as long as it is a subset of the array, which means that a single array element and the entire array can both be considered slabs. A slab is defined by the multidimensional coordinate of its initial vertex and the lengths of each dimension.
Given this description of the slab concept, the usage of the remaining arguments should become apparent. The argument start is a one-dimensional array specifying the location in the SDS array at which the write operation will begin. The values of each element of the array start are relative to 0 in both the C and FORTRAN-77 interfaces. The size of start must be the same as the number of dimensions in the SDS array. In addition, each value in start must be smaller than its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return
FAIL.The argument stride is a one-dimensional array specifying, for each dimension, the interval between values to be written. For example, setting the first element of the array stride equal to 1 writes data to every location along the first dimension. Setting the first element of the array stride to 2 writes data to every other location along the first dimension. Figure 3b illustrates this example, where the shading elements are written and the white elements are skipped. If the argument stride is set to
NULLin C (or either0or1in FORTRAN-77), SDwritedata operates as if every element of stride contains a value of 1, and a contiguous write is performed. For better performance, it is recommended that the value of stride be defined asNULL(i.e.,0or1in FORTRAN-77) rather than being set to 1.The size of the array stride must be the same as the number of dimensions in the SDS array. Also, each value in stride must be smaller than or equal to its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return
FAIL.FIGURE 3b An Example of Access Pattern ("Strides")The argument edges is a one-dimensional array specifying the length of each dimension of the slab to be written. If the slab has fewer dimensions than the SDS data set has, the size of edges must still be equal to the number of dimensions in the SDS array and all the elements corresponding to the additional dimensions must be set to 1.
Each value in the array edges must not be larger than the length of the corresponding dimension in the SDS data set unless the dimension is unlimited. Attempting to write slabs larger than the size of the SDS data set will result in an error condition.
In addition, the sum of each value in the array edges and the corresponding value in the start array must be smaller than or equal to its corresponding SDS array dimension unless the dimension is unlimited. Violating any of these conditions causes SDwritedata to return
FAIL. When SDreaddata returns FAIL (or -1) due to any invalid argements, the error codeDFE_ARGSwill be pushed on the stack.The parameter data contains the SDS data to be written. If the SDS array is smaller than the buffer data, the amount of data written will be limited to the maximum size of the SDS array.
Be aware that the mapping between the dimensions of a slab and the order in which the slab values are stored in memory is different between C and FORTRAN-77. In C, the values are stored with the assumption that the last dimension of the slab varies fastest (or "row-major order" storage), but in FORTRAN-77 the first dimension varies fastest (or "column-major order" storage). These storage order conventions can cause some confusion when data written by a C program is read by a FORTRAN-77 program or vice versa.
There are two FORTRAN-77 versions of this routine: sfwdata and sfwcdata. The routine sfwdata writes numeric scientific data and sfwcdata writes character scientific data.
SDwritedata returns either a value of
SUCCEED(or0) orFAIL(or-1). The parameters of this routine are described in Table 3D.TABLE 3D SDwritedata Parameter List3.5.1.1 Filling an Entire Array
Filling an array is a simple slab operation where the slab begins at the origin of the SDS array and fills every location in the array. SDwritedata fills an entire SDS array with data when all elements of the array start are set to
0, the argument stride is set equal toNULLin C or each element of the array stride is set to1in both C and FORTRAN-77, and each element of the array edges is equal to the length of each dimension.EXAMPLE 2. Writing to an SDS.This example illustrates the use of the routines SDselect/sfselect and SDwritedata/sfwrite to select the first SDS in the file SDS.hdf created in Example 1 and to write actual data to it.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" #define X_LENGTH 5 #define Y_LENGTH 16 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 start[2], edges[2]; int32 data[Y_LENGTH][X_LENGTH]; int i, j; /********************* End of variable declaration ***********************/ /* * Data set data initialization. */ for (j = 0; j < Y_LENGTH; j++) { for (i = 0; i < X_LENGTH; i++) data[j][i] = (i + j) + 1; } /* * Open the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_WRITE); /* * Attach to the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Define the location and size of the data to be written to the data set. */ start[0] = 0; start[1] = 0; edges[0] = Y_LENGTH; edges[1] = X_LENGTH; /* * Write the stored data to the data set. The third argument is set to NULL * to specify contiguous data elements. The last argument must * be explicitly cast to a generic pointer since SDwritedata is designed * to write generic data. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)data); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program write_data implicit none C C Parameter declaration. C character*7 FILE_NAME character*11 SDS_NAME integer X_LENGTH, Y_LENGTH, RANK parameter (FILE_NAME = 'SDS.hdf', + SDS_NAME = 'SDStemplate', + X_LENGTH = 5, + Y_LENGTH = 16, + RANK = 2) integer DFACC_WRITE, DFNT_INT32 parameter (DFACC_WRITE = 2, + DFNT_INT32 = 24) C C Function declaration. C integer sfstart, sfselect, sfwdata, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index, status integer start(2), edges(2), stride(2) integer i, j integer data(X_LENGTH, Y_LENGTH) C C**** End of variable declaration ************************************ C C C Data set data initialization. C do 20 j = 1, Y_LENGTH do 10 i = 1, X_LENGTH data(i, j) = i + j - 1 10 continue 20 continue C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_WRITE) C C Attach to the first data set. C sds_index = 0 sds_id = sfselect(sd_id, sds_index) C C Define the location and size of the data to be written C to the data set. Note that setting values of the array stride to 1 C specifies the contiguous writing of data. C start(1) = 0 start(2) = 0 edges(1) = X_LENGTH edges(2) = Y_LENGTH stride(1) = 1 stride(2) = 1 C C Write the stored data to the data set named in SDS_NAME. C Note that the routine sfwdata is used instead of sfwcdata C to write the numeric data. C status = sfwdata(sds_id, start, stride, edges, data) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.5.1.2 Writing Slabs to an SDS Array
To allow preexisting data to be modified, the HDF library does not prevent SDwritedata from overwriting one slab with another. As a result, the calling program is responsible for managing any overlap when writing slabs. The HDF library will issue an error if a slab extends past the valid boundaries of the SDS array. However, appending data along an unlimited dimension is allowed.
EXAMPLE 3. Writing a Slab of Data to an SDS.This example shows how to fill a 3-dimensional SDS array with data by writing series of 2-dimensional slabs to it.
C:#include "mfhdf.h" #define FILE_NAME "SLABS.hdf" #define SDS_NAME "FilledBySlabs" #define X_LENGTH 4 #define Y_LENGTH 5 #define Z_LENGTH 6 #define RANK 3 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id; intn status; int32 dim_sizes[3], start[3], edges[3]; int32 data[Z_LENGTH][Y_LENGTH][X_LENGTH]; int32 zx_data[Z_LENGTH][X_LENGTH]; int i, j, k; /********************* End of variable declaration ***********************/ /* * Data initialization. */ for (k = 0; k < Z_LENGTH; k++) for (j = 0; j < Y_LENGTH; j++) for (i = 0; i < X_LENGTH; i++) data[k][j][i] = (i + 1) + (j + 1) + (k + 1); /* * Create the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_CREATE); /* * Define dimensions of the array to be created. */ dim_sizes[0] = Z_LENGTH; dim_sizes[1] = Y_LENGTH; dim_sizes[2] = X_LENGTH; /* * Create the array with the name defined in SDS_NAME. */ sds_id = SDcreate (sd_id, SDS_NAME, DFNT_INT32, RANK, dim_sizes); /* * Set the parameters start and edges to write * a 6x4 element slab of data to the data set; note * that edges[1] is set to 1 to define a 2-dimensional slab * parallel to the ZX plane. * start[1] (slab position in the array) is initialized inside * the for loop. */ edges[0] = Z_LENGTH; edges[1] = 1; edges[2] = X_LENGTH; start[0] = start[2] = 0; for (j = 0; j < Y_LENGTH; j++) { start[1] = j; /* * Initialize zx_data buffer (data slab). */ for ( k = 0; k < Z_LENGTH; k++) { for ( i = 0; i < X_LENGTH; i++) { zx_data[k][i] = data[k][j][i]; } } /* * Write the data slab into the SDS array defined in SDS_NAME. * Note that the 3rd parameter is NULL which indicates that consecutive * slabs in the Y direction are written. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)zx_data); } /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program write_slab implicit none C C Parameter declaration. C character*9 FILE_NAME character*13 SDS_NAME integer X_LENGTH, Y_LENGTH, Z_LENGTH, RANK parameter (FILE_NAME = 'SLABS.hdf', + SDS_NAME = 'FilledBySlabs', + X_LENGTH = 4, + Y_LENGTH = 5, + Z_LENGTH = 6, + RANK = 3) integer DFACC_CREATE, DFNT_INT32 parameter (DFACC_CREATE = 4, + DFNT_INT32 = 24) C C Function declaration. C integer sfstart, sfcreate, sfwdata, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id integer dim_sizes(3), start(3), edges(3), stride(3) integer i, j, k, status integer data(X_LENGTH, Y_LENGTH, Z_LENGTH) integer xz_data(X_LENGTH, Z_LENGTH) C C**** End of variable declaration ************************************ C C C Data initialization. C do 30 k = 1, Z_LENGTH do 20 j = 1, Y_LENGTH do 10 i = 1, X_LENGTH data(i, j, k) = i + j + k 10 continue 20 continue 30 continue C C Create the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_CREATE) C C Define dimensions of the array to be created. C dim_sizes(1) = X_LENGTH dim_sizes(2) = Y_LENGTH dim_sizes(3) = Z_LENGTH C C Create the data set with the name defined in SDS_NAME. C sds_id = sfcreate(sd_id, SDS_NAME, DFNT_INT32, RANK, . dim_sizes) C C Set the parameters start and edges to write C a 4x6 element slab of data to the data set; C note that edges(2) is set to 1 to define a 2 dimensional slab C parallel to the XZ plane; C start(2) (slab position in the array) is initialized inside the C for loop. C edges(1) = X_LENGTH edges(2) = 1 edges(3) = Z_LENGTH start(1) = 0 start(3) = 0 stride(1) = 1 stride(2) = 1 stride(3) = 1 do 60 j = 1, Y_LENGTH start(2) = j - 1 C C Initialize the buffer xz_data (data slab). C do 50 k = 1, Z_LENGTH do 40 i = 1, X_LENGTH xz_data(i, k) = data(i, j, k) 40 continue 50 continue C C Write the data slab into SDS array defined in SDS_NAME. C Note that the elements of array stride are set to 1 to C specify that the consecutive slabs in the Y direction are written. C status = sfwdata(sds_id, start, stride, edges, xz_data) 60 continue C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) endEXAMPLE 4. Altering Values within an SDS Array.This example demonstrates how the routine SDwritedata can be used to alter the values of the elements in the 10th and 11th rows, at the 2nd column, in the SDS array created in the Example 1 and written in Example 2. FORTRAN-77 routine sfwdata is used to alter the elements in the 2nd row, 10th and 11th columns, to reflect the difference between C and Fortran internal storage.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 start[2], edges[2]; int32 new_data[2]; int i, j; /********************* End of variable declaration ***********************/ /* * Open the file and initialize the SD interface with write access. */ sd_id = SDstart (FILE_NAME, DFACC_WRITE); /* * Select the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Set up the start and edge parameters to write new element values * into 10th row, 2nd column place, and 11th row, 2nd column place. */ start[0] = 9; /* starting at 10th row */ start[1] = 1; /* starting at 2nd column */ edges[0] = 2; /* rows 10th and 11th */ edges[1] = 1; /* column 2nd only */ /* * Initialize buffer with the new values to be written. */ new_data[0] = new_data[1] = 1000; /* * Write the new values. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)new_data); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program alter_data implicit none C C Parameter declaration. C character*7 FILE_NAME integer DFACC_WRITE parameter (FILE_NAME = 'SDS.hdf', + DFACC_WRITE = 2) C C Function declaration. C integer sfstart, sfselect, sfwdata, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index integer start(2), edges(2), stride(2) integer status integer new_data(2) C C**** End of variable declaration ************************************ C C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_WRITE) C C Select the first data set. C sds_index = 0 sds_id = sfselect(sd_id, sds_index) C C Initialize the start, edge, and stride parameters to write C two elements into 2nd row, 10th column and 11th column places. C C Specify 2nd row. C start(1) = 1 C C Specify 10th column. C start(2) = 9 edges(1) = 1 C C Two elements are written along 2nd row. C edges(2) = 2 stride(1) = 1 stride(2) = 1 C C Initialize the new values to be written. C new_data(1) = 1000 new_data(2) = 1000 C C Write the new values. C status = sfwdata(sds_id, start, stride, edges, new_data) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.5.1.3 Appending Data to an SDS Array along an Unlimited Dimension
An SDS array can be made appendable, however, only along one dimension. This dimension must be specified as an appendable dimension when it is created.
In C, only the first element of the SDcreate parameter dim_sizes (i.e., the dimension of the lowest rank or the slowest-changing dimension) can be assigned the value
SD_UNLIMITED(or0) to make the first dimension unlimited. In FORTRAN-77, only the last dimension (i.e., the dimension of the highest rank or the slowest-changing dimension) can be unlimited. In other words, in FORTRAN-77 dim_sizes(rank) must be set to the valueSD_UNLIMITEDto make the last dimension appendable.To append data to a data set without overwriting previously-written data, the user must specify the appropriate coordinates in the start parameter of the SDwritedata routine. For example, if 15 data elements have been written to an unlimited dimension, appending data to the array requires a start coordinate of 15. Specifying a starting coordinate less than the current number of elements written to the unlimited dimension will result in data being overwritten. In either case, all of the coordinates in the array except the one corresponding to the unlimited dimension must be equal to or less than the lengths of their corresponding dimensions.
Any time an unlimited dimension is appended to, the HDF library will automatically adjust the dimension record to the new length. If the newly-appended data begins beyond the previous length of the dimension, the locations between the old data and the beginning of the newly-appended data are initialized to the assigned fill value if there is one defined by the user, or the default fill value if none is defined. Refer to Section 3.10.5 on page 106 for a discussion of fill value.
3.5.1.4 Determining whether an SDS Array is Appendable: SDisrecord
SDisrecord determines whether the data set identified by the parameter sds_id is appendable, which means that the slowest-changing dimension of the SDS array is declared unlimited when the data set is created. The syntax of SDisrecord is as follows:
SDisrecord returns
TRUE(or1) when the data set specified by sds_id is appendable andFALSE(or0) otherwise. The parameter of this routine is defined in Table 3E.TABLE 3E SDisrecord Parameter List
3.5.1.5 Setting the Block Size: SDsetblocksize
SDsetblocksize sets the size of the blocks used for storing the data for unlimited dimension data sets. This is used only when creating new data sets; it does not have any affect on existing data sets. The syntax of this routine is as follows:
SDsetblocksize must be called after SDcreate or SDselect and before SDwritedata. The parameter block_size should be set to a multiple of the desired buffer size.
SDsetblocksize returns a value of
SUCCEED(or0) orFAIL(or-1). Its parameters are further described in Table 3F.3.5.1.6 Setting the I/O Access Type of an SDS: SDsetaccesstype
SDsetaccesstype sets the type of I/O (serial, parallel,...) for accessing the data of the data set identified by sds_id. Valid values of access_types are
DFACC_SERIAL(or1),DFACC_PARALLEL(or11), andDFACC_DEFAULT(or0.) The syntax of this routine is as follows:SDsetaccesstype returns a value of
SUCCEED(or0) if the SDS data can be accessed via accesstype orFAIL(or-1) otherwise. Its parameters are further described in Table 3F.TABLE 3F SDsetblocksize and SDsetaccesstype Parameter ListEXAMPLE 5. Appending Data to an SDS Array with an Unlimited Dimension.This example creates a 10x10 SDS array with one unlimited dimension and writes data to it. The file is reopened and the routine SDisrecord/sfisrcrd is used to determine whether the selected SDS array is appendable. Then new data is appended, starting at the 11th row.
C:#include "mfhdf.h" #define FILE_NAME "SDSUNLIMITED.hdf" #define SDS_NAME "AppendableData" #define X_LENGTH 10 #define Y_LENGTH 10 #define RANK 2 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 dim_sizes[2]; int32 data[Y_LENGTH][X_LENGTH], append_data[X_LENGTH]; int32 start[2], edges[2]; int i, j; /********************* End of variable declaration ***********************/ /* * Data initialization. */ for (j = 0; j < Y_LENGTH; j++) { for (i = 0; i < X_LENGTH; i++) data[j][i] = (i + 1) + (j + 1); } /* * Create the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_CREATE); /* * Define dimensions of the array. Make the first dimension * appendable by defining its length to be unlimited. */ dim_sizes[0] = SD_UNLIMITED; dim_sizes[1] = X_LENGTH; /* * Create the array data set. */ sds_id = SDcreate (sd_id, SDS_NAME, DFNT_INT32, RANK, dim_sizes); /* * Define the location and the size of the data to be written * to the data set. */ start[0] = start[1] = 0; edges[0] = Y_LENGTH; edges[1] = X_LENGTH; /* * Write the data. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)data); /* * Terminate access to the array data set, terminate access * to the SD interface, and close the file. */ status = SDendaccess (sds_id); status = SDend (sd_id); /* * Store the array values to be appended to the data set. */ for (i = 0; i < X_LENGTH; i++) append_data[i] = 1000 + i; /* * Reopen the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_WRITE); /* * Select the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Check if selected SDS is unlimited. If it is not, then terminate access * to the SD interface and close the file. */ if ( SDisrecord (sds_id) ) { /* * Define the location of the append to start at the first column * of the 11th row of the data set and to stop at the end of the * eleventh row. */ start[0] = Y_LENGTH; start[1] = 0; edges[0] = 1; edges[1] = X_LENGTH; /* * Append data to the data set. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)append_data); } /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program append_sds implicit none C C Parameter declaration. C character*16 FILE_NAME character*14 SDS_NAME integer X_LENGTH, Y_LENGTH, RANK parameter (FILE_NAME = 'SDSUNLIMITED.hdf', + SDS_NAME = 'AppendableData', + X_LENGTH = 10, + Y_LENGTH = 10, + RANK = 2) integer DFACC_CREATE, DFACC_WRITE, SD_UNLIMITED, + DFNT_INT32 parameter (DFACC_CREATE = 4, + DFACC_WRITE = 2, + SD_UNLIMITED = 0, + DFNT_INT32 = 24) C C Function declaration. C integer sfstart, sfcreate, sfwdata, sfselect integer sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index, status integer dim_sizes(2) integer start(2), edges(2), stride(2) integer i, j integer data (X_LENGTH, Y_LENGTH), append_data(X_LENGTH) C C**** End of variable declaration ************************************ C C C Data initialization. C do 20 j = 1, Y_LENGTH do 10 i = 1, X_LENGTH data(i, j) = i + j 10 continue 20 continue C C Create the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_CREATE) C C Define dimensions of the array. Make the C last dimension appendable by defining its length as unlimited. C dim_sizes(1) = X_LENGTH dim_sizes(2) = SD_UNLIMITED C Create the array data set. sds_id = sfcreate(sd_id, SDS_NAME, DFNT_INT32, RANK, . dim_sizes) C C Define the location and the size of the data to be written C to the data set. Note that the elements of array stride are C set to 1 for contiguous writing. C start(1) = 0 start(2) = 0 edges(1) = X_LENGTH edges(2) = Y_LENGTH stride(1) = 1 stride(2) = 1 C C Write the data. C status = sfwdata(sds_id, start, stride, edges, data) C C Terminate access to the data set, terminate access C to the SD interface, and close the file. C status = sfendacc(sds_id) status = sfend(sd_id) C C Store the array values to be appended to the data set. C do 30 i = 1, X_LENGTH append_data(i) = 1000 + i - 1 30 continue C C Reopen the file and initialize the SD. C sd_id = sfstart(FILE_NAME, DFACC_WRITE) C C Select the first data set. C sds_index = 0 sds_id = sfselect(sd_id, sds_index) C C Define the location of the append to start at the 11th C column of the 1st row and to stop at the end of the 10th row. C start(1) = 0 start(2) = Y_LENGTH edges(1) = X_LENGTH edges(2) = 1 C C Append the data to the data set. C status = sfwdata(sds_id, start, stride, edges, append_data) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.5.2 Compressing SDS Data: SDsetcompress
The SDsetcompress routine compresses an existing data set or creates a new compressed data set. It is a simplified interface to the HCcreate routine, and should be used instead of HCcreate unless the user is familiar with the lower-level routines.
The compression algorithms currently supported by SDsetcompress are:
The syntax of the routine SDsetcompress is as follows:
The parameter comp_type specifies the compression type definition and is set to
COMP_CODE_NONE(or 0) for no compressionCOMP_CODE_RLE(or 1) for run-length encoding (RLE)COMP_CODE_SKPHUFF(or 3) for Skipping HuffmanCOMP_CODE_DEFLATE(or 4) for GZIP compressionCOMP_CODE_SZIP(or 5) for Szip compressionCompression information is specified by the parameter c_info in C, and by the parameter comp_prm in FORTRAN-77. The parameter c_info is a pointer to a union structure of type comp_info. Refer to the SDsetcompress entry in the HDF Reference Manual for the description of the comp_info structure.
If comp_type is set to
COMP_CODE_NONEorCOMP_CODE_RLE, the parameters c_info and comp_prm are not used; c_info can be set toNULLand comp_prm can be undefined.If comp_type is set to
COMP_CODE_SKPHUFF, then the structure skphuff in the union comp_info in C (comp_prm(1) in FORTRAN-77) must be provided with the size, in bytes, of the data elements.If comp_type is set to
COMP_CODE_DEFLATE, the deflate structure in the union comp_info in C (comp_prm(1) in FORTRAN-77) must be provided with the information about the compression effort.If comp_type is set to
COMP_CODE_SZIP, the Szip options mask and the number of pixels per block in a chunked and Szip-compressed dataset must be specified inc_info.szip.options_maskandc_info.szip.pixels_per_blockin C, and comp_prm(1) and comp_prm(2) in Fortran, respectively.For example, to compress signed 16-bit integer data using the adaptive Huffman algorithm, the following definition and SDsetcompress call are used.
To compress a data set using the gzip deflation algorithm with the maximum effort specified, the following definition and SDsetcompress call are used.
SDsetcompress functionality is currently limited to the following:
- Write the compressed data, in its entirety, to the data set. The data set is built in-core then written in a single write operation.
- Append to a compressed data set. The data of the data set is read into memory, appended with data along the unlimited dimension, then compressed and written back to the data set.
The existing compression algorithms supported by HDF do not allow partial modification to a compressed datastream. Overwriting the contents of existing data sets may be supported in the future. Note also that SDsetcompress performs the compression of the data, not SDwritedata.
SDsetcompress returns a value of
SUCCEED(or0) orFAIL(or-1). The C version parameters are further described in Table 3G and the FORTRAN-77 version parameters are further described in Table 3H.
TABLE 3G SDsetcompress Parameter ListTABLE 3H sfscompress Parameter List
This example uses the routine SDsetcompress/sfscompress to compress SDS data with the GZIP compression method. See comments in the program regarding the use of the Skipping Huffman or RLE compression methods.
C:#include "mfhdf.h" #define FILE_NAME "SDScompressed.hdf" #define SDS_NAME "SDSgzip" #define X_LENGTH 5 #define Y_LENGTH 16 #define RANK 2 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 comp_type; /* Compression flag */ comp_info c_info; /* Compression structure */ int32 start[2], edges[2], dim_sizes[2]; int32 data[Y_LENGTH][X_LENGTH]; int i, j; /********************* End of variable declaration ***********************/ /* * Buffer array data and define array dimensions. */ for (j = 0; j < Y_LENGTH; j++) { for (i = 0; i < X_LENGTH; i++) data[j][i] = (i + j) + 1; } dim_sizes[0] = Y_LENGTH; dim_sizes[1] = X_LENGTH; /* * Create the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_CREATE); /* * Create the data set with the name defined in SDS_NAME. */ sds_id = SDcreate (sd_id, SDS_NAME, DFNT_INT32, RANK, dim_sizes); /* * Ininitialize compression structure element and compression * flag for GZIP compression and call SDsetcompress. * * To use the Skipping Huffman compression method, initialize * comp_type = COMP_CODE_SKPHUFF * c_info.skphuff.skp_size = value * * To use the RLE compression method, initialize * comp_type = COMP_CODE_RLE * No structure element needs to be initialized. */ comp_type = COMP_CODE_DEFLATE; c_info.deflate.level = 6; status = SDsetcompress (sds_id, comp_type, &c_info); /* * Define the location and size of the data set * to be written to the file. */ start[0] = 0; start[1] = 0; edges[0] = Y_LENGTH; edges[1] = X_LENGTH; /* * Write the stored data to the data set. The last argument * must be explicitly cast to a generic pointer since SDwritedata * is designed to write generic data. */ status = SDwritedata (sds_id, start, NULL, edges, (VOIDP)data); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program write_compressed_data implicit none C C Parameter declaration. C character*17 FILE_NAME character*7 SDS_NAME integer X_LENGTH, Y_LENGTH, RANK parameter (FILE_NAME = 'SDScompressed.hdf', + SDS_NAME = 'SDSgzip', + X_LENGTH = 5, + Y_LENGTH = 16, + RANK = 2) integer DFACC_CREATE, DFNT_INT32 parameter (DFACC_CREATE = 4, + DFNT_INT32 = 24) integer COMP_CODE_DEFLATE parameter (COMP_CODE_DEFLATE = 4) integer DEFLATE_LEVEL parameter (DEFLATE_LEVEL = 6) C To use Skipping Huffman compression method, declare C integer COMP_CODE_SKPHUFF C parameter(COMP_CODE_SKPHUFF = 3) C To use RLE compression method, declare C integer COMP_CODE_RLE C parameter(COMP_CODE_RLE = 1) C C C Function declaration. C integer sfstart, sfcreate, sfwdata, sfendacc, sfend, + sfscompress C C**** Variable declaration ******************************************* C integer sd_id, sds_id, status integer start(2), edges(2), stride(2), dim_sizes(2) integer comp_type integer comp_prm(1) integer data(X_LENGTH, Y_LENGTH) integer i, j C C**** End of variable declaration ************************************ C C C Buffer array data and define array dimensions. C do 20 j = 1, Y_LENGTH do 10 i = 1, X_LENGTH data(i, j) = i + j - 1 10 continue 20 continue dim_sizes(1) = X_LENGTH dim_sizes(2) = Y_LENGTH C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_CREATE) C C Create the data set with the name SDS_NAME. C sds_id = sfcreate(sd_id, SDS_NAME, DFNT_INT32, RANK, dim_sizes) C C Initialize compression parameter (deflate level) C and call sfscompress function C For Skipping Huffman compression, comp_prm(1) should be set C to skipping sizes value (skp_size). C comp_type = COMP_CODE_DEFLATE comp_prm(1) = deflate_level status = sfscompress(sds_id, comp_type, comp_prm(1)) C C Define the location and size of the data that will be written to C the data set. C start(1) = 0 start(2) = 0 edges(1) = X_LENGTH edges(2) = Y_LENGTH stride(1) = 1 stride(2) = 1 C C Write the stored data to the data set. C status = sfwdata(sds_id, start, stride, edges, data) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.5.3 External File Operations
The HDF library provides routines to store SDS arrays in an external file that is separate from the primary file containing the metadata for the array. Such an SDS array is called an external SDS array. With external arrays, it is possible to link data sets in the same HDF file to multiple external files or data sets in different HDF files to the same external file.
External arrays are functionally identical to arrays in the primary data file. The HDF library keeps track of the beginning of the data set and adds data at the appropriate position in the external file. When data is written or appended along a specified dimension, the HDF library writes along that dimension in the external file and updates the appropriate dimension record in the primary file.
There are two methods for creating external SDS arrays. The user can create a new data set in an external file or move data from an existing internal data set to an external file. In either case, only the array values are stored externally, all metadata remains in the primary HDF file.
When an external array is created, a sufficient amount of space is reserved in the external file for the entire data set. The data set will begin at the specified byte offset and extend the length of the data set. The write operation will overwrite the target locations in the external file. The external file may be of any format, provided the data types, byte ordering, and dimension ordering are supported by HDF. However, the primary file must be an HDF file.
Routines for manipulating external SDS arrays can only be used with HDF files. Unidata-formatted netCDF files are not supported by these routines.
3.5.3.1 Specifying the Directory Search Path of an External File: HXsetdir
There are three filesystem locations the HDF external file routines check when determining the location of an external file. They are, in order of search precedence:
The syntax of HXsetdir is as follows:
HXsetdir has one argument, a string specifying the directory list to be searched. This list can consist of one directory name or a set of directory names separated by colons. The FORTRAN-77 version of this routine takes an additional argument, dir_length, which specifies the length of the directory list string.
If an error condition is encountered, HXsetdir leaves the directory search path unchanged. The directory search path specified by HXsetdir remains in effect throughout the scope of the calling program.
HXsetdir returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of HXsetdir are described in Table 3I on page 54.3.5.3.2 Specifying the Location of the Next External File to be Created: HXsetcreatedir
HXsetcreatedir specifies the directory location of the next external file to be created. It overrides the directory location specified by $HDFEXTCREATEDIR and the locations searched by the open(3) call in the same manner as HXsetdir. Specifically, the search precedence is:
The syntax of HXsetcreatedir is as follows:
HXsetcreatedir has one argument, the directory location of the next external file to be created. The FORTRAN-77 version of this routine takes an additional argument, dir_length, which specifies the length of the directory list string. If an error is encountered, the directory location is left unchanged.
HXsetcreatedir returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of HXsetcreatedir are described in Table 3I.TABLE 3I HXsetdir and HXsetcreatedir Parameter Lists3.5.3.3 Creating a Data Set with Data Stored in an External File: SDsetexternalfile
Creating a data set in an external file involves the following steps:
To create a data set with data stored in an external file, the calling program must make the following calls.
C: sds_id = SDcreate(sd_id, name, data_type, rank, dim_sizes);
status = SDsetexternalfile(sds_id, filename, offset);
status = SDwritedata(sds_id, start, stride, edges, data);
status = SDendaccess(sds_id);
FORTRAN: sds_id = sfcreate(sd_id, name, data_type, rank, dim_sizes)
status = sfsextf(sds_id, filename, offset)
status = sfwdata(sds_id, start, stride, edges, data)
OR status = sfwcdata(sds_id, start, stride, edges, data)
status = sfendacc(sds_id)
For a newly-created data set, SDsetexternalfile marks the SDS identified by sds_id as one whose data is to be written to an external file. It does not actually write data to an external file; it marks the data set as an external data set for all subsequent SDwritedata operations.
Note that data can only be moved once for any given data set, i.e., SDsetexternalfile can only be called once after a data set has been created. It is the user's responsibility to make sure that the external data file is kept with the primary HDF file.
The parameter filename is the name of the external data file and offset is the number of bytes from the beginning of the external file to the location where the first byte of data should be written. If a file with the name specified by filename exists in the current directory search path, HDF will access it as the external file. If the file does not exist, HDF will create one in the directory named in the last call to HXsetcreatefile. If an absolute pathname is specified, the external file will be created at the location specified by the pathname, overriding the location specified by the last call to HXsetcreatefile. Use caution when writing to existing external or primary files since the HDF library starts the write operation at the specified offset without determining whether data is being overwritten.
Once the name of an external file is established, it cannot be changed without breaking the association between the data set's metadata and the data it describes.
SDsetexternalfile returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDsetexternalfile are described in Table 3J.TABLE 3J SDsetexternalfile Parameter List3.5.3.4 Moving Existing Data to an External File
Data can be moved from a primary file to an external file. The following steps perform this task:
To move data set data to an external file, the calling program must make the following calls:
For an existing data set, SDsetexternalfile moves the data to the external file. Any data in the external file that occupies the space reserved for the external array will be overwritten as a result of this operation. Data of an existing data set in the primary file can only be moved to the external file once. During the operation, the data is written to the external file as a contiguous stream regardless of how it is stored in the primary file. Because data is moved as is, any unwritten locations in the data set are preserved in the external file. Subsequent read and write operations performed on the data set will access the external file.
EXAMPLE 7. Moving Data to the External File.This example illustrates the use of the routine SDsetexternalfile/sfsextf to move the SDS data written in Example 2 to the external file.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" #define EXT_FILE_NAME "ExternalSDS" #define OFFSET 24 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index, offset; intn status; /********************* End of variable declaration ***********************/ /* * Open the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_WRITE); /* * Select the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Create a file with the name EXT_FILE_NAME and move the data set * values into it, starting at byte location OFFSET. */ status = SDsetexternalfile (sds_id, EXT_FILE_NAME, OFFSET); /* * Terminate access to the data set, SD interface, and file. */ status = SDendaccess (sds_id); status = SDend (sd_id); }FORTRAN:program write_extfile implicit none C C Parameter declaration. C character*7 FILE_NAME character*11 EXT_FILE_NAME integer OFFSET integer DFACC_WRITE parameter (FILE_NAME = 'SDS.hdf', + EXT_FILE_NAME = 'ExternalSDS', + OFFSET = 24, + DFACC_WRITE = 2) C C Function declaration. C integer sfstart, sfselect, sfsextf, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index, offset integer status C C**** End of variable declaration ************************************ C C C Open the HDF file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_WRITE) C C Select the first data set. C sds_index = 0 sds_id = sfselect(sd_id, sds_index) C C Create a file with the name EXT_FILE_NAME and move the data set C into it, starting at byte location OFFSET. C status = sfsextf(sds_id, EXT_FILE_NAME, OFFSET) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.6 Reading Data from an SDS Array: SDreaddata
Data of an SDS array can be read as an entire array, a subset of the array, or a set of samples of the array. SDS data is read from an external file in the same way that it is read from a primary file; whether the SDS array is stored in an external file is transparent to the user. Reading data from an SDS array involves the following steps:
To read data from an SDS array, the calling program must contain the following function calls:
Note that step 2 is not illustrated in the function call syntax; it is carried out by assigning values to the parameters start, stride, and edges before the routine SDreaddata is called in step 3.
SDreaddata reads the data according to the definition specified by the parameters start, stride, and edges and stores the data into the buffer provided, data. The argument sds_id is the SDS identifier returned by SDcreate or SDselect. As with SDwritedata, the arguments start, stride, and edges describe the starting location, the number of elements to skip after each read, and the number of elements to be read, respectively, for each dimension. For additional information on the parameters start, stride, and edges, refer to Section 3.5.1 on page 31.
There are two FORTRAN-77 versions of this routine: sfrdata reads numeric data and sfrcdata reads character data.
SDreaddata returns a value of
SUCCEED(or0), including the situation when the data set does not contain data, orFAIL(or-1). The parameters of SDreaddata are further described in Table 3K.TABLE 3K SDreaddata Parameter ListEXAMPLE 8. Reading from an SDS.This example uses the routine SDreaddata/sfrdata to read the data that has been written in Example 2, modified in Example 4, and moved to the external file in the Example 7. Note that the original file SDS.hdf that contains the SDS metadata and the external file ExternalSDS that contains the SDS raw data should reside in the same directory. The fact that raw data is in the external file is transparent to the user's program.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" #define X_LENGTH 5 #define Y_LENGTH 16 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 start[2], edges[2]; int32 data[Y_LENGTH][X_LENGTH]; int i, j; /********************* End of variable declaration ***********************/ /* * Open the file for reading and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_READ); /* * Select the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Set elements of array start to 0, elements of array edges * to SDS dimensions,and use NULL for the argument stride in SDreaddata * to read the entire data. */ start[0] = 0; start[1] = 0; edges[0] = Y_LENGTH; edges[1] = X_LENGTH; /* * Read entire data into data array. */ status = SDreaddata (sds_id, start, NULL, edges, (VOIDP)data); /* * Print 10th row; the following numbers should be displayed. * * 10 1000 12 13 14 */ for (j = 0; j < X_LENGTH; j++) printf ("%d ", data[9][j]); printf ("\n"); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program read_data implicit none C C Parameter declaration. C character*7 FILE_NAME integer X_LENGTH, Y_LENGTH parameter (FILE_NAME = 'SDS.hdf', + X_LENGTH = 5, + Y_LENGTH = 16) integer DFACC_READ, DFNT_INT32 parameter (DFACC_READ = 1, + DFNT_INT32 = 24) C C Function declaration. C integer sfstart, sfselect, sfrdata, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index, status integer start(2), edges(2), stride(2) integer data(X_LENGTH, Y_LENGTH) integer j C C**** End of variable declaration ************************************ C C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_READ) C C Select the first data set. C sds_index = 0 sds_id = sfselect(sd_id, sds_index) C C Set elements of the array start to 0, elements of the array edges to C SDS dimensions, and elements of the array stride to 1 to read the C entire data. C start(1) = 0 start(2) = 0 edges(1) = X_LENGTH edges(2) = Y_LENGTH stride(1) = 1 stride(2) = 1 C C Read entire data into data array. Note that sfrdata is used C to read the numeric data. C status = sfrdata(sds_id, start, stride, edges, data) C C Print 10th column; the following numbers are displayed: C C 10 1000 12 13 14 C write(*,*) (data(j,10), j = 1, X_LENGTH) C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) endEXAMPLE 9. Reading Subsets of an SDS.This example shows how parameters start, stride, and edges of the routine SDreadata/sfrdata can be used to read three subsets of an SDS array.
C:
For the first subset, the program reads every 3rd element of the 2nd column starting at the 4th row of the data set created in Example 2 and modified in Examples 4 and 7.For the second subset the program reads the first 4 elements of the 10th row.For the third subset, the program reads from the same data set every 6th element of each column and 4th element of each row starting at 1st column, 3d row.FORTRAN-77:
Fortran program reads transposed data to reflect the difference in C and Fortran internal storage.C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" #define SUB1_LENGTH 5 #define SUB2_LENGTH 4 #define SUB3_LENGTH1 2 #define SUB3_LENGTH2 3 main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id, sds_index; intn status; int32 start[2], edges[2], stride[2]; int32 sub1_data[SUB1_LENGTH]; int32 sub2_data[SUB2_LENGTH]; int32 sub3_data[SUB3_LENGTH2][SUB3_LENGTH1]; int i, j; /********************* End of variable declaration ***********************/ /* * Open the file for reading and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_READ); /* * Select the first data set. */ sds_index = 0; sds_id = SDselect (sd_id, sds_index); /* * Reading the first subset. * * Set elements of start, edges, and stride arrays to read * every 3rd element in the 2nd column starting at 4th row. */ start[0] = 3; /* 4th row */ start[1] = 1; /* 2nd column */ edges[0] = SUB1_LENGTH; /* SUB1_LENGTH elements are read along 2nd column*/ edges[1] = 1; stride[0] = 3; /* every 3rd element is read along 2nd column */ stride[1] = 1; /* * Read the data from the file into sub1_data array. */ status = SDreaddata (sds_id, start, stride, edges, (VOIDP)sub1_data); /* * Print what we have just read; the following numbers should be displayed: * * 5 8 1000 14 17 */ for (j = 0; j < SUB1_LENGTH; j++) printf ("%d ", sub1_data[j]); printf ("\n"); /* * Reading the second subset. * * Set elements of start and edges arrays to read * first 4 elements of the 10th row. */ start[0] = 9; /* 10th row */ start[1] = 0; /* 1st column */ edges[0] = 1; edges[1] = SUB2_LENGTH; /* SUB2_LENGTH elements are read along 10th row */ /* * Read data from the file into sub2_data array. Note that the third * parameter is set to NULL for contiguous reading. */ status = SDreaddata (sds_id, start, NULL, edges, (VOIDP)sub2_data); /* * Print what we have just read; the following numbers should be displayed: * * 10 1000 12 13 */ for (j = 0; j < SUB2_LENGTH; j++) printf ("%d ", sub2_data[j]); printf ("\n"); /* * Reading the third subset. * * Set elements of the arrays start, edges, and stride to read * every 6th element in the column and 4th element in the row * starting at 1st column, 3d row. */ start[0] = 2; /* 3d row */ start[1] = 0; /* 1st column */ edges[0] = SUB3_LENGTH2; /* SUB3_LENGTH2 elements are read along each column */ edges[1] = SUB3_LENGTH1; /* SUB3_LENGTH1 elements are read along each row */ stride[0] = 6; /* read every 6th element along each column */ stride[1] = 4; /* read every 4th element along each row */ /* * Read the data from the file into sub3_data array. */ status = SDreaddata (sds_id, start, stride, edges, (VOIDP)sub3_data); /* * Print what we have just read; the following numbers should be displayed: * * 3 7 * 9 13 * 15 19 */ for ( j = 0; j < SUB3_LENGTH2; j++ ) { for (i = 0; i < SUB3_LENGTH1; i++) printf ("%d ", sub3_data[j][i]); printf ("\n"); } /* * Terminate access to the data set. */ status = SDendaccess (sds_id); /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program read_subsets implicit none C C Parameter declaration. C character*7 FILE_NAME parameter (FILE_NAME = 'SDS.hdf') integer DFACC_READ, DFNT_INT32 parameter (DFACC_READ = 1, + DFNT_INT32 = 24) integer SUB1_LENGTH, SUB2_LENGTH, SUB3_LENGTH1, + SUB3_LENGTH2 parameter (SUB1_LENGTH = 5, + SUB2_LENGTH = 4, + SUB3_LENGTH1 = 2, + SUB3_LENGTH2 = 3) C C Function declaration. C integer sfstart, sfselect, sfrdata, sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id, sds_index, status integer start(2), edges(2), stride(2) integer sub1_data(SUB1_LENGTH) integer sub2_data(SUB2_LENGTH) integer sub3_data(SUB3_LENGTH1,SUB3_LENGTH2) integer i, j C C**** End of variable declaration ************************************ C C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_READ) C C Select the first data set. C sds_index = 0 sds_id =sfselect(sd_id, sds_index) C C Reading the first subset. C C Set elements of start, stride, and edges arrays to read C every 3d element in in the 2nd row starting in the 4th column. C start(1) = 1 start(2) = 3 edges(1) = 1 edges(2) = SUB1_LENGTH stride(1) = 1 stride(2) = 3 C C Read the data from sub1_data array. C status = sfrdata(sds_id, start, stride, edges, sub1_data) C C Print what we have just read, the following numbers should be displayed: C C 5 8 1000 14 17 C write(*,*) (sub1_data(j), j = 1, SUB1_LENGTH) C C Reading the second subset. C C Set elements of start, stride, and edges arrays to read C first 4 elements of 10th column. C start(1) = 0 start(2) = 9 edges(1) = SUB2_LENGTH edges(2) = 1 stride(1) = 1 stride(2) = 1 C C Read the data into sub2_data array. C status = sfrdata(sds_id, start, stride, edges, sub2_data) C C Print what we have just read; the following numbers should be displayed: C C 10 1000 12 13 C write(*,*) (sub2_data(j), j = 1, SUB2_LENGTH) C C Reading the third subset. C C Set elements of start, stride and edges arrays to read C every 6th element in the row and every 4th element in the column C starting at 1st row, 3rd column. C start(1) = 0 start(2) = 2 edges(1) = SUB3_LENGTH1 edges(2) = SUB3_LENGTH2 stride(1) = 4 stride(2) = 6 C C Read the data from the file into sub3_data array. C status = sfrdata(sds_id, start, stride, edges, sub3_data) C C Print what we have just read; the following numbers should be displayed: C C 3 9 15 C 7 13 19 C do 50 i = 1, SUB3_LENGTH1 write(*,*) (sub3_data(i,j), j = 1, SUB3_LENGTH2) 50 continue C C Terminate access to the data set. C status = sfendacc(sds_id) C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.7 Obtaining Information about SD Data Sets
The routines covered in this section provide methods for obtaining information about all scientific data sets in a file, for identifying the data sets that meet certain criteria, and for obtaining information about specific data sets.
SDfileinfo obtains the numbers of data sets and file attributes, set by SD interface routines, in a file. SDgetinfo provides information about an individual SDS. To retrieve information about all data sets in a file, a calling program can use SDfileinfo to determine the number of data sets, followed by repeated calls to SDgetinfo to obtain the information about a particular data set.
SDnametoindex, SDnametoindices, or SDreftoindex can be used to obtain the index of an SDS in a file knowing its name or reference number. Refer to Section 3.2.1 on page 20 for a description of the data set index and reference number. SDidtoref is used when the reference number of an SDS is required by another routine and the SDS identifier is available.
These routines are described individually in the following subsections.
3.7.1 Obtaining Information about the Contents of a File: SDfileinfo
SDfileinfo determines the number of scientific data sets and the number of file attributes contained in a file. This information is often useful in index validation or sequential searches. The syntax of SDfileinfo is as follows:
SDfileinfo stores the numbers of scientific data sets and file attributes in the parameters n_datasets and n_file_attrs, respectively. Note that the value returned by n_datasets will include the number of SDS arrays and the number of dimension scales. Refer to Section 3.8.4 on page 79 and Section 3.8.4.4 on page 86 for the description of dimension scales and its association with SDS arrays as well as how to distinguish between SDS arrays and dimension scales. The file attributes are those that are created by SDsetattr for an SD interface identifier instead of an SDS identifier. Refer to Section 3.9.1 on page 91 for the discussion of SDsetattr.
SDfileinfo returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDfileinfo are specified in Table 3L on page 67.3.7.2 Obtaining Information about a Specific SDS: SDgetinfo
SDgetinfo provides basic information about an SDS array. Often information about an SDS array is needed before reading and working with the array. For instance, the rank, dimension sizes, and/or data type of an array are needed to allocate the proper amount of memory to work with the array. SDgetinfo takes an SDS identifier as input, and retrieves the name, rank, dimension sizes, data type, and number of attributes for the corresponding SDS. The syntax of this routine is as follows:
SDgetinfo stores the name, rank, dimension sizes, data type, and number of attributes of the specified data set into the parameters sds_name, rank, dim_sizes, data_type, and n_attrs, respectively. The parameter sds_name is a character string. Note that the name of the SDS is limited to
64characters.If the data set is created with an unlimited dimension, then in the C interface, the first element of the dim_sizes array (corresponding to the slowest-changing dimension) contains the number of records in the unlimited dimension; in the FORTRAN-77 interface, the last element of the array dim_sizes (corresponding to the slowest-changing dimension) contains this information.
The parameter data_type contains any type that HDF supports for the scientific data. Refer to Table 2F on page 14, for the list of supported data types and their corresponding defined values. The parameter n_attrs only reflects the number of attributes assigned to the data set specified by sds_id; file attributes are not included. Use SDfileinfo to get the number of file attributes.
SDgetinfo returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDgetinfo are specified in Table 3L on page 67.3.7.3 Obtaining Data Set Compression Information: SDgetcompinfo
SDgetcompinfo retrieves the compression information used to create or write an SDS data set. SDgetcompinfo replaces SDgetcompress because this function has flaws, causing failure for some chunked and chunked/compressed data.
The compression algorithms currently available for SDS data set are as follows:
SDgetcompinfo takes one input parameter, sds_id, a data set identifier, and two return parameters, comp_type, identifying the type of compression used, and either c_info (in C) or comp_prm (in FORTRAN-77), containing further compression information.
The syntax of SDgetcompinfo is as follows:
See Section 3.5.2, "Compressing SDS Data: SDsetcompress," for a discussion of comp_type, c_info, ane comp_prm, and a list of supported compression modes.
The parameter comp_type specifies the compression type definition and is set to
COMP_CODE_NONE(or 0) for no compressionCOMP_CODE_RLE(or 1) for run-length encoding (RLE)COMP_CODE_SKPHUFF(or 3) for Skipping HuffmanCOMP_CODE_DEFLATE(or 4) for GZIP compressionCOMP_CODE_SZIP(or 5) for Szip compressionCompression information is returned by the parameter c_info in C, and by the parameter comp_prm in FORTRAN-77. The parameter c_info is a pointer to a union structure of type comp_info. Refer to the SDsetcompress entry in the HDF Reference Manual for the description of the comp_info structure.)
When comp_type is
COMP_CODE_NONEorCOMP_CODE_RLE, the parameters c_info and comp_prm are unchanged.When comp_type is
COMP_CODE_SKPHUFF, then the structure skphuff in the union comp_info in C (comp_prm(1) in FORTRAN-77) will store the size, in bytes, of the data elements.When comp_type is
COMP_CODE_DEFLATE, then the deflate structure in the union comp_info in C (comp_prm(1) in FORTRAN-77) will store the information about the compression effort.When comp_type is
COMP_CODE_SZIP, then the Szip options mask and the number of pixels per block in a chunked and Szip-compressed dataset will be specified inc_info.szip.options_maskandc_info.szip.pixels_per_blockin C, and comp_prm(1) and comp_prm(2) in Fortran, respectively.SDgetcompinfo returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDgetcompinfo are specified in Table 3L.TABLE 3L SDfileinfo, SDgetinfo, and SDgetcompinfo Parameter ListsEXAMPLE 10. Getting Information about a File and an SDSs.This example illustrates the use of the routine SDfileinfo/sffinfo to obtain the number of data sets in the file SDS.hdf and the routine SDgetinfo/sfginfo to retrieve the name, rank, dimension sizes, data type and number of attributes of the selected data set.
C:#include "mfhdf.h" #define FILE_NAME "SDS.hdf" main( ) { /************************* Variable declaration **************************/ int32 sd_id, sds_id; intn status; int32 n_datasets, n_file_attrs, index; int32 dim_sizes[MAX_VAR_DIMS]; int32 rank, data_type, n_attrs; char name[MAX_NC_NAME]; int i; /********************* End of variable declaration ***********************/ /* * Open the file and initialize the SD interface. */ sd_id = SDstart (FILE_NAME, DFACC_READ); /* * Determine the number of data sets in the file and the number * of file attributes. */ status = SDfileinfo (sd_id, &n_datasets, &n_file_attrs); /* * Access every data set and print its name, rank, dimension sizes, * data type, and number of attributes. * The following information should be displayed: * * name = SDStemplate * rank = 2 * dimension sizes are : 16 5 * data type is 24 * number of attributes is 0 */ for (index = 0; index < n_datasets; index++) { sds_id = SDselect (sd_id, index); status = SDgetinfo (sds_id, name, &rank, dim_sizes, &data_type, &n_attrs); printf ("name = %s\n", name); printf ("rank = %d\n", rank); printf ("dimension sizes are : "); for (i=0; i< rank; i++) printf ("%d ", dim_sizes[i]); printf ("\n"); printf ("data type is %d\n", data_type); printf ("number of attributes is %d\n", n_attrs); /* * Terminate access to the data set. */ status = SDendaccess (sds_id); } /* * Terminate access to the SD interface and close the file. */ status = SDend (sd_id); }FORTRAN:program get_data_set_info implicit none C C Parameter declaration. C character*7 FILE_NAME parameter (FILE_NAME = 'SDS.hdf') integer DFACC_READ, DFNT_INT32 parameter (DFACC_READ = 1, + DFNT_INT32 = 24) integer MAX_NC_NAME, MAX_VAR_DIMS parameter (MAX_NC_NAME = 256, + MAX_VAR_DIMS = 32) C C Function declaration. C integer sfstart, sffinfo, sfselect, sfginfo integer sfendacc, sfend C C**** Variable declaration ******************************************* C integer sd_id, sds_id integer n_datasets, n_file_attrs, index integer status, n_attrs integer rank, data_type integer dim_sizes(MAX_VAR_DIMS) character name *(MAX_NC_NAME) integer i C C**** End of variable declaration ************************************ C C C Open the file and initialize the SD interface. C sd_id = sfstart(FILE_NAME, DFACC_READ) C C Determine the number of data sets in the file and the number of C file attributes. C status = sffinfo(sd_id, n_datasets, n_file_attrs) C C Access every data set in the file and print its name, rank, C dimension sizes, data type, and number of attributes. C The following information should be displayed: C C name = SDStemplate C rank = 2 C dimension sizes are : 5 16 C data type is 24 C number of attributes is 0 C do 10 index = 0, n_datasets - 1 sds_id = sfselect(sd_id, index) status = sfginfo(sds_id, name, rank, dim_sizes, data_type, . n_attrs) write(*,*) "name = ", name(1:15) write(*,*) "rank = ", rank write(*,*) "dimension sizes are : ", (dim_sizes(i), i=1, rank) write(*,*) "data type is ", data_type write(*,*) "number of attributes is ", n_attrs C C Terminate access to the current data set. C status = sfendacc(sds_id) 10 continue C C Terminate access to the SD interface and close the file. C status = sfend(sd_id) end3.7.4 Locating an SDS by Name: SDnametoindex
SDnametoindex determines and returns the index of a data set in a file given the data set's name. The syntax of this routine is as follows:
The parameter sds_name is a character string with the maximum length of
64characters. If more than one data set has the name specified by sds_name, SDnametoindex will return the index of the first data set, which could be an SDS or a coordinate variable (also called dimension scale.) Note that if there are more than one data set with the same name in the file, writing to a data set returned by this function without verifying that it is the desired data set could cause data corruption. Refer to the Important Note on page 3-21 for more details regarding the problem and how to handle it.SDgetnumvars_byname can be used to get the number of data sets (or variables, which includes both data sets and coordinate variables) with the same name. SDnametoindices can be used to get a list of structures containing the indices and the types of all the variables of that same name.
An index obtained by SDnametoindex or SDnametoindices can then be used by SDselect to obtain an SDS identifier for the specified data set. The SDnametoindex routine is case-sensitive to the name specified by sds_name and does not accept wildcards as part of that name. The name must exactly match the name of the SDS being searched for.
SDnametoindex returns the index of a data set or
FAIL(or-1). The parameters of SDnametoindex are specified in Table 3M on page 71.3.7.5 Locating More Than One SDS by the Same Name: SDnametoindices
SDnametoindices returns indices of all data sets having the same name. The data sets can be either SDSs or coordinate variables. The syntax of this routine is as follows:
The parameter sds_name is a character string with the maximum length of
64characters.SDnametoindices retrieves a list of structures
varlist_t, containing the indices and the types of all variables of the same name sds_name. The structurevarlist_tis defined as:typedef struct varlist{int32 var_index; /* index of a variable */vartype_t var_type; /* type of a variable */} varlist_t;The type of a variable
vartype_tis defined as:IS_SDSVAR=0 : variable is an actual SDSIS_CRDVAR=1 : variable is a coordinate variableUNKNOWN=2 : variable is created before HDF4.2r2, unknown typePrior to calling SDnametoindices, SDgetnumvars_byname can be used to get the number of data sets, with which the application can allocate var_list appropriately. Also, when the number of data sets returned is 1, the application can call SDnametoindex instead of SDnametoindices for simplicity.
An index obtained by SDnametoindex or SDnametoindices can then be used by SDselect to obtain an SDS identifier for the specified data set.
The SDnametoindices routine is case-sensitive to the name specified by sds_name and does not accept wildcards as part of that name. The name must match exactly the name of the SDS being searched for.
SDnametoindices returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDnametoindices are specified in Table 3M on page 71.3.7.6 Getting Number of Data Sets Given a Name: SDgetnumvars_byname
SDgetnumvars_byname determines and returns the number of variables in a file having the same name. The variables may include both data sets and coordinate variables. The syntax of this routine is as follows:
The parameter sds_name is a character string with the maximum length of
64characters.SDgetnumvars_byname returns a value of
SUCCEED(or0) orFAIL(or-1). The parameters of SDgetnumvars_byname are specified in Table 3M on page 71.TABLE 3M SDnametoindex, SDnametoindices, and SDgetnumvars_byname Parameter Lists3.7.7 Locating an SDS by Reference Number: SDreftoindex
SDreftoindex determines and returns the index of a data set in a file given the data set's reference number. The syntax of this routine is as follows:
The reference number can be obtained using SDidtoref if the SDS identifier is available. Remember that reference numbers do not necessarily adhere to any ordering scheme.
SDreftoindex returns either the index of an SDS or
FAIL(or-1). The parameters of this routine are specified in Table 3N on page 73.3.7.8 Obtaining the Reference Number Assigned to the Specified SDS: SDidtoref
SDidtoref returns the reference number of the data set identified by the parameter sds_id if the data set is found, or
FAIL(or-1) otherwise. The syntax of this routine is as follows:This reference number is often used by Vaddtagref to add the data set to a vgroup. Refer to Chapter 5, Vgroups (V API), for more information.
The parameter of SDidtoref is specified in Table 3N on page 73.
3.7.9 Obtaining the Type of an HDF4 Object: SDidtype
SDidtype returns the type of an object, given the object's identifier, obj_id. The syntax of this routine is as follows:
SDidtype returns a value of type hdf_idtype_t, which can be one of the following:
NOT_SDAPI_ID(or-1)SD_ID(or0)SDS_ID(or1)DIM_ID(or2)
SDidtype returns
NOT_SDAPI_IDfor either when obj_id is not a valid HDF identifier, or is a valid HDF identifier, but not one of the identifier types in the SD interface, which are SD identifier, SDS identifier, and dimension identifier.The parameter of SDidtype is specified in Table 3N.
3.7.10 Determining whether an SDS is empty: SDcheckempty
SDcheckempty takes an SDS identifier, sds_id, as input, and returns a single parameter indicating whethe
