Using OPeNDAP Clients to Visualize HDF-EOS2 Grid and Swath Data

Introduction

We have tested the use of several OPeNDAP client tools to visualize NASA HDF4 data via OPeNDAP with the HDF4-OPeNDAP data handler(version 3.7.14 or earlier). Unfortunately, most of them failed to display any images with HDF-EOS2 data. After in-depth investigations, we found that it is mainly because these clients follow CF conventions and DAP output doesn't necessarily follow CF conventions. After finding the reason, we then added two configuration options in the HDF4 handler to partially support CF conventions with the help of either the StructMetadata parser (--enable-cf) or the HDF-EOS2 library (--enable-cf and --with-hdfeos2). We hope this will provide a way to make several clients visualize some HDF-EOS2 data. This CF option sacrifices some features provided by the default HDF4 handler but it is useful for one to visualize NASA HDF data with existing OPeNDAP clients. We've tested our CF configuration option with the following clients on NASA HDF-EOS2 Grid and Swath data:

Our goal of this experiment is to provide information on how one can use the enhanced HDF4-OPeNDAP handler to access some NASA HDF-EOS products by the OPeNDAP client tools listed above. MODIS and AIRS data products are used to test with the enhanced data handler.

OPeNDAP Visualization Client Problem that Follows CF Convention

In general, OPeNDAP does not have restrictions on the length of variable name, the format in attributes and the format of DAP Grid. However, some OPeNDAP visualization clients follow strictly CF conventions such as:

Variable name should be 15 characters or less.
Shared dimensions must be defined and available outside DAP Grid.
Shared dimensions must have pre-defined units in attributes.
Array within a DAP Grid should have a name for each matching dimension.

Proposed Limited Solution

We provide --enable-cf and the optional --use-hdfeos2 configuration option. When you build hdf4_handler, we strongly suggest you to enable both --enable-cf and --use-hdfeos2 because using HDF-EOS2 library will maximize the coverage of HDF-EOS2 data that can be accessed by the enhanced handler. Our solution will affect the behavior of hdf4_handler in the following ways:

It won't process HDF4 Vgroup.
It will generate independent shared dimension variable outside of Grid.
Some attributes and variables are renamed (e.g., XDim into lon) and inserted (i.e.,Conventions "COARDS, GrADS").
It will adjust the dimensions and datasets to support non-standard Grid projection like Polar and Sinusoidal.

The following figures illustrate how our solution (--enable-cf and --with-hdfeos2) affects the DAP output. Figure 1b shows that the StructMetadata attribute is suppressed under the CF option. Figure 2b shows that the DAP Grid structure is generated instead of the plain DAP array as shown in Figure 2a.

Figure 1a: Default HDF4 Handler DAS Output

Figure 1b: HDF4 Handler DAS Output with the CF option enabled

Figure 2a: Default HDF4 Handler DDS Output

Figure 2b: HDF4 Handler DDS Output withh the CF option enabled

Detailed Behaviors of Each Client after Applying Our Solution

The following is our current experience with the OPeNDAP visualization clients. The test HDF-EOS2 data files are

IDV

IDV is very sensitive to attributes (i.e. DAS information) returned by OPeNDAP. It parses DAS information first and then retrieves DDS information from an OPeNDAP server. If a variable doesn't show up in DAS, the IDV GUI client doesn't display the variable even if DDS has it. It requires shared dimension variables defined outside DAP Grids and precise unit/value pair attributes associated with them. For example, units "degrees_north", "degrees_east" and "level" must be provided to display data properly.

The below is a screen-shot of a AQUA AIRS Grid file.

The below is a screen-shot of a AQUA MODIS Grid file.

IDV can also display a Swath data with 2-D lat/lon map data. The below is a screen-shot of a AQUA AIRS Swath file.

Panoply

Panoply is a Java based tool like IDV but it can display Grid only.

The below is a screen-shot of a AQUA AIRS Grid file.

The below is a screen-shot of a AQUA MODIS Grid file.

OpenGrADS

OpenGrADS required to have named dimensions in Array within a Grid data-type.

The below is a screen-shot of a AQUA AIRS Grid file.

The below is a screen-shot of a AQUA MODIS Grid file.

Here's a short summary of OPeNDAP visualization clients:

Client	Problem
IDV	N/A
Panoply	No support for Swath
OpenGrADS	No support for Swath

Limitations and Potential Problems

Please see Limitations page.

Conclusion

Although some clients can display Grid data, many failed to display Swath. At this stage, many HDF-EOS2 files cannot be visualized through the new enhanced HDF4 handler due to the difference of CF model model and HDF-EOS2 data model. A new HDF4 handler is needed to do the job.

- - Last modified:September 23rd 2009