hdf images hdf images

HOME > PROJECTS > NOAA-SDS

Demo

This page will explain how to use HDF4-to-HDF5 utility to convert HDF-EOS2 files into netCDF-4-compliant HDF5 files.

Prerequisite

This project is included in H4H5TOOLS 2.0 (or higher). By default, HDF4-to-HDF5 utility does not convert semantically correct HDF5 file from an HDF-EOS2 file. H4H5TOOLS 2.0 (or higher) should be configured with --with-hdfeos2 option. This procedure is explained in the install document in the package.

Input file: HDF-EOS2 file

NSIDC has lots of HDF-EOS2 files. You can use the following link to download one HDF-EOS2 file. ftp://n4ftl01u.ecs.nasa.gov/SAN/AMSA/AE_RnGd.001/2007.07.01/AMSR_E_L3_RainGrid_B05_200707.hdf external

You can browse the input file by using HDFView with HDF-EOS Plug-in external . The following two screenshots show how the input file looks like:

Execute HDF4-to-HDF5 Utility

HDF4-to-HDF5 Utility can convert an HDF4 file into an HDF5 file. Two additional options -eos and -nc4strict will turn on additional features from this project.

The following command will convert an HDF-EOS2 file AMSR_E_L3_RainGrid_B05_200707.hdf into a netCDF-4-compliant HDF5 file AMSR_E_L3_RainGrid_B05_200707.h5.

$ h4toh5 -eos -nc4strict AMSR_E_L3_RainGrid_B05_200707.hdf AMSR_E_L3_RainGrid_B05_200707.h5

Output file: HDF-EOS2 file

You can browse the output file by using HDFView or ncdump that is part of netCDF-4 external. The abridged result of ncdump is the following: 

netcdf AMSR_E_L3_RainGrid_B05_200707 {
group: MonthlyRainTotal_GeoGrid {

// global attributes:
	    :GridYear = 2007 ;
	    :GridMonth = 7 ;
	    :TbOceanRain_description = "Brightness temperature derived monthly rain total over ocean." ;
	    :RrLandRain_description = "Rain rate derived monthly rain total over land." ;

group: Data\ Fields {
dimensions:
    lon = 72 ;
    lat = 28 ;
variables:
    float RrLandRain(lat, lon) ;
    float TbOceanRain(lat, lon) ;
    double lat(lat) ;
    double lon(lon) ;
} // group Data\ Fields
}
Both RrLandRain and TbOceanRain, and four attributes were converted. Also, two coordinate variables lon and lat were generated. All values in these two coordinate variables are generated from the projection code in the given HDF-EOS2 file.

Verification

ncdump

ncdump shows the structure, attributes and variables. To check whether or not the generated file is netCDF-4-compliant, ncdump can be used. Also, the result can be compared with the result of h5dump that is part of HDF5 library.

Verifier

This utility can verify the correctness by comparing all values from the input file and the output file. Currently, it only works for HDF-EOS2 Grid data. You can get more information and download it from Download page.

This utility receives two filenames: an HDF-EOS2 file and a netCDF-4-compliant HDF5 file.

$ ./verify AMSR_E_L3_RainGrid_B05_200707.hdf AMSR_E_L3_RainGrid_B05_200707.h5

This command should not print any error unless these two files are different. You can put additional flags such as -p and -v to see verbose messages.

NCL

If you want visual output, you can use NCAR Command Language (NCL) external. NCL can read both HDF-EOS2 file and netCDF-4 file although it has restrictions. To draw charts with NCL, you need to write .ncl files for the HDF-EOS2 file and the netCDF-4-compliant HDF5 file respectively.

HDF-EOS2 file

NCL document states that NCL does not handle geolocation information. So, longitude and latitude should be manually augmented; otherwise, NCL does not find corresponding coordinates. Also, field names are different because NCL flattens the hierarchy and appends Grid data name to field names. AE_RnGd.hdfeos2.ncl is an example NCL file that draws charts. With this description file, you can get two charts as the following:

netCDF-4-compliant HDF5 file

Since NCL 5.0.0 supports only netCDF classic model, netCDF-4 group is not supported. Since HDF4-to-HDF5 converter tries to keep its original hierarchy as much as possible, however, the converted files have group. To make the converted file read by NCL, the converted file should be flattened manually.

dump

ncdump is part of netCDF-4 package, and the output of ncdump can be saved as .cdl file.

$ ncdump AMSR_E_L3_RainGrid_B05_200707.h5 > AMSR_E_L3_RainGrid_B05_200707.cdl

edit

You can open the AMSR_E_L3_RainGrid_B05_200707.cdl file with your favorite text editor. It will look like the following: 

netcdf AMSR_E_L3_RainGrid_B05_200707 {

// global attributes:
	    :HDFEOSVersion_GLOSDS = "HDFEOS_V2.13" ;

group: MonthlyRainTotal_GeoGrid {
...
group: Data\ Fields {
dimensions:
    lon = 72 ;
    lat = 28 ;
variables:
    float TbOceanRain(lat, lon) ;
	    TbOceanRain:HDF4_OBJECT_TYPE = "SDS" ;
	    TbOceanRain:HDF4_OBJECT_NAME = "TbOceanRain" ;
    double lon(lon) ;
	    lon:long_name = "longitude" ;
	    lon:units = "degrees_east" ;
    double lat(lat) ;
	    lat:long_name = "latitude" ;
	    lat:units = "degrees_north" ;
    float RrLandRain(lat, lon) ;
	    RrLandRain:HDF4_OBJECT_TYPE = "SDS" ;
	    RrLandRain:HDF4_OBJECT_NAME = "RrLandRain" ;
...
}
}
As you can see, it has two groups, and they should be removed because the classic model does not allow groups.

Remove hierarchy so that all dimensions and variables reside in the root as the following shows: 

netcdf modified {
dimensions:
  lon = 72 ;
  lat = 28 ;
variables:
  float TbOceanRain(lat, lon) ;
	  TbOceanRain:HDF4_OBJECT_TYPE = "SDS" ;
	  TbOceanRain:HDF4_OBJECT_NAME = "TbOceanRain" ;
  double lon(lon) ;
	  lon:long_name = "longitude" ;
	  lon:units = "degrees_east" ;
  double lat(lat) ;
	  lat:long_name = "latitude" ;
	  lat:units = "degrees_north" ;
  float RrLandRain(lat, lon) ;
	  RrLandRain:HDF4_OBJECT_TYPE = "SDS" ;
	  RrLandRain:HDF4_OBJECT_NAME = "RrLandRain" ;

// global attributes:
	  :HDF4_OBJECT_TYPE = "Vgroup" ;
	  :HDF4_OBJECT_NAME = "Data Fields" ;
data:

TbOceanRain =
...
}

generate

With this modified .cdl file, you can generate netCDF-4 file by using ncgen. ncgen is also part of netCDF-4 package. In the following command, -v3 was given to generate netCDF-4/HDF5 file.

$ ncgen -v3 AMSR_E_L3_RainGrid_B05_200707.cdl -o AMSR_E_L3_RainGrid_B05_200707.h5.nc

The generated file can be ready by NCL.

Unlike the HDF-EOS2, longitude and latitude do not need to be augmented. Pointing lon and lat in the .ncl file will make NCL recognize them. AE_RnGd.netcdf4.ncl is an example NCL file that draws charts. You can get two charts from this description file.

These results are equivalent to results from the HDF-EOS2 file.

- - Last modified:September 23rd 2009