Ted Habermann, The HDF Group
Fifteen years ago, NASA selected HDF as the format for the data products produced by NASA Satellites for the NASA Earth Observing System (EOS).
The HDF Earth Science Program is well aware of this important legacy. We focus on continuing support of U.S. environmental satellite programs (NASA Earth Observing System and Joint Polar Satellite System, JPSS), on-going quality assurance of the HDF libraries and helping data users access and understand products written in HDF. The HDF-EOS Information Center (#hdfeos) includes code examples in MATLAB, IDL, NCL, and Python, many driven by user questions. The site also provides information on other HDF tools.
NASA’s decision ensured a role for HDF in Earth Science and set an important precedent. HDF developers, along with the U.S. and other Earth Observing nations, developed a clear distinction between Earth Science Data Objects (grids, swaths, profiles…); the metadata required to describe them; and the HDF objects (datasets, groups, attributes, etc.) that make them up.
The critical realization was that communities like EOS needed conventions for describing Earth Science objects to enable using and sharing those objects. These conventions, termed HDF-EOS, have been used successfully in hundreds of NASA products that can be easily shared among multiple users using standard tools.
Many other Earth Science communities have used the powerful combination of conventions and HDF. A well-known example is Unidata’s netCDF. It started as a file format and then became an API to HDF that supports the Unidata Common Data Model (CDM). The NetCDF4 API to HDF is used throughout the Ocean and Atmospheric Science communities and is the standard selected for the next U.S. geostationary satellite program (GOES-R). Another example is the Bathymetric Attributed Grid (BAG) format used by The Open Navigation Surface project, NOAA’s National Ocean Service and the International Hydrographic Organization (IHO).
Historically, the HDF Earth Science Group has organized the roughly annual HDF Workshop. Until 2013, this workshop was focused primarily on NASA users. Since 2013, the workshop has been co-located with the summer Earth Science Information Partners (ESIP) meeting. This has expanded the audience considerably to include HDF users from many communities. Presentations from over 17 years of workshops are available on slideshare. They provide a very interesting historic reference on HDF development and other data related topics.
The HDF Earth Science Group has extended its reach recently to include strong contributions to adoption of international metadata standards in NASA and other communities as well as tools and guidance for evaluating and improving metadata for Earth Science products. We are also developing new tools for designing and browsing HDF products. Stay tuned to the HDF Blog for details.