HDF5 Known Problems

Problems Reported as of HDF5 1.8.7

Conversion Tests fail on Mac OS X 10.7 (Lion)

Users have reported that when building HDF5, the conversion tests failed (make check) in dt_arith.chk.

A workaround is to edit ./< HDF5source >/config/gnu-flags, search for PROD_CFLAGS under "gcc-4.*", and change the value for PROD_CFLAGS to "-O0".

Problems Reported in HDF5 1.8.6

HDFView: Unsupported fileformat error opening an HDF/HDF5 file on Mac OS X

If you are on Mac OS X and attempt to open an HDF/HDF5 file in HDFView 2.7, and it fails with this error,

  HDF file :"Failed to open file xxx.hdf java.io.IOException: Unsupported fileformat - xxx.hdf"

then you may be running the wrong version of HDFView for your system.

If you get this error with the 64-bit version of HDFView, then try the 32-bit version. If you get this error for the 32-bit version, then try the 64-bit version. See the HDFView Home page to obtain the 32-bit or 64-bit version of HDFView for Mac OS X.

Problem Reading A Collectively Written Dataset In Parallel

While working on the 1.8.6 release of HDF5, a bug was discovered that can occur when reading from a dataset in parallel shortly after it has been written to collectively. The issue was exposed by a new test in the parallel HDF5 test suite, but had existed before that. We believe the problem lies with certain MPI implementations and/or filesystems.

We have provided a pure MPI test program, as well as a standalone HDF5 program, that can be used to determine if this is an issue on your system. They should be run across multiple nodes with a varying number of processes. These programs can be found at:

  http://www.hdfgroup.org/ftp/HDF5/examples/known_problems/

Although it does not affect the above issue, this is a good time to point out that the HDF5 library does not make any attempt to guarantee sequential consistency of MPI I/O calls. It is the application's responsibility to enforce sequential consistency as needed, as described in section 13.6.1 of the MPI Standard. If sequential consistency is needed, the application should close the file, issue an MPI_Barrier(), then reopen the file before and after each collective write.

It is also possible to guarantee sequential consistency using MPI_File_set_atomicity(), however there is currently no way to do this within HDF5. We are planning to add an API function to allow users to set the atomicity in a future release.

HDF5 1.8.5 Problem Affecting Files with Non-default Addresses

A problem was discovered in HDF5 1.8.5 that affects files with non-default sizes of addresses and lengths as set by H5Pset_sizes(). The problem occurs when (sizeof_addr + 2*sizeof_size) is not a multiple of 8, unless using the new file format (H5Pset_libver_bounds()). Files created under these conditions by 1.8.5 will be readable by 1.8.5 but not by any other version of HDF5. Files created under these conditions by other versions of HDF5 will not be readable by 1.8.5.

We recommend that those who use H5Pset_sizes or may deal with files that use this feature move to the latest snapshot (snap4) of HDF5 1.8.5. If you have files created by 1.8.5 under these conditions that you wish to keep, you can fix this problem by using "h5repack -L" with the 1.8.5 version of h5repack to recreate the file with the new file format. You can then read this file with any version since 1.8.0, and if desired use h5repack (without -L) again to move back to the old file format.

Please see the HDF Downloads page for obtaining HDF5 releases and snapshots.

HDFView 2.6.1 Problem on Some Windows XP Machines

HDFView 2.6.1 works properly on most Windows XP machines. However, on others, HDF5 is greyed out under the Help menu, and HDF5 is not supported. This occurs if a required package is missing on that machine. A user reported that .NET 3.5 Service Pack 1 is the missing package. This can be obtained from here:

 http://download.microsoft.com/download/2/0/e/20e90413-712f-438c-988e-fdaa79a8ac3d/dotnetfx35.exe

HDFView 2.6.1 Does Not Display the User's Guide on Macintosh

On the Macintosh, you cannot access the User's Guide in HDFView 2.6.1 by either selecting it under the Help menu bar or clicking on the small book icon. The workaround is to access the on-line version for now. This is located here:

   http://www.hdfgroup.org/hdf-java-html/hdfview/UsersGuide/index.html

In HDFView, you can select the Tools -> User Options pull-down menu item, to modify the path to the User's Guide to point to the on-line version. This will work while HDFView is running, but once it is closed, this information will be lost.

Corruption Problem In HDF5 1.8.0 through HDF5 1.8.4

A corruption problem was found in the HDF5 1.8 release, which affects versions 1.8.0 through 1.8.4. The problem has been fixed in HDF5 1.8.4 Patch 1.

What Causes The Problem

Files that have this corruption problem meet all of the following circumstances:

The version of HDF5 was before the HDF5 1.8.4 Patch 1 release (which includes the 1.8.4 release).
The file was created on a big-endian platform (SPARC/Solaris, POWER/AIX, etc).
NetCDF-4 was used to create the file, OR the H5Pset_libver_bounds(fapl, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST) call was made, OR shared object header messages were enabled with H5Pset_shared_mesg_nindexes() when creating the file.
More than 8 attributes were added to an object in the file (in the case of using netCDF-4 or calling H5Pset_libver_bounds), OR if messages of the type specified to be shared were stored in the file (in the case of calling H5Pset_shared_mesg_nindexes).

If your data matches these criteria, then you may have generated files that have incorrectly encoded IDs for attributes and shared object header messages.

How To Determine If Your Files Are Corrupted

There is no foolproof way to detect a file with these IDs incorrectly encoded, but if the h5dump tool reports an error message similar to the following when displaying information about the file, it is likely that you have a file with this problem:

      h5dump error: error getting attribute information

For files created with netCDF-4, this message will generally display when dumping information about the root group. If your application is generating an error stack similar to this, it is also likely that the file has this problem:

       .
       .
       .
       #008: ../../hdf5_v1.8/src/H5B2int.c line 1951 in H5B2_iterate_node():
   iterator function failed major: B-Tree node minor: Unable to list node
       #009: ../../hdf5_v1.8/src/H5Adense.c line 1076 in
   H5A_dense_iterate_bt2_cb(): heap op callback failed major: Attribute minor:
   Can't operate on object
       #010: ../../hdf5_v1.8/src/H5HF.c line 680 in H5HF_op(): can't operate on
   object from fractal heap major: Heap minor: Can't operate on object
       #011: ../../hdf5_v1.8/src/H5HFman.c line 462 in H5HF_man_op(): unable to
   operate on heap object major: Heap minor: Can't operate on object
       #012: ../../hdf5_v1.8/src/H5HFman.c line 276 in H5HF_man_op_real():
   fractal heap object offset too large major: Heap minor: Out of range

Fixing The Problem

A tool is being developed that can be run on a corrupted file to correct it.

Please contact the HDF Helpdesk if you are unable to wait for this tool.

Problems Reported in HDF5 1.8.3

HDF5-1.8.3 and HDFView 2.5 always modify an HDF5 file opened in read/write mode

HDF5-1.8.3 and HDFView 2.5 always re-wrote the superblock of an HDF5 file with a later version of the superblock if the file was opened for read-write access. Therefore an HDF5 file would inadvertently be upgraded to a later version of the file format. This issue was fixed in HDF5-1.8.4.

CFLAGS Getting Overwritten by HDF5

A user should be able to set CFLAGS and have those options be used to build HDF5. However, HDF5 sets CFLAGS in the Makefiles when it should not, so a user cannot do this. We are planning on fixing this for the HDF5 1.8.4 release.

Problems Reported in HDF5 1.8.2

The dt_arith conversion test fails with gcc

Optimization does not handle conversion well with gcc 3.* and 4.*.

To get around the problem, turn off optimization. You can do this by editting the ./config/gnu-flags file in the HDF5 source code and setting PROD_CFLAGS from "-O" to "-O0".

Here is the code that needs to be modified:

     gcc-3.[0-4]*|gcc-4.[0123]*)
        # The optimization level is reduced for gcc 3.* and 4.* due to problems
        # with code generation for src/H5Tconv.c with the -O2 & -O3
        # optimization levels (which shows up as failures for various integer
        # types -> long long conversions in the test/dtypes test).  Perhaps
        # later versions of gcc will fix this bug... - QAK - 2003/10/20
        PROD_CFLAGS="-O"
        ;;

HDFView: Dataset Displayed Incorrectly When Viewed As Image

With HDFView 2.5, a dataset that is not an image gets displayed incorrectly when viewed as an image. This does not occur if the dataset is an image to begin with. If you select "Open As" and choose to view a dataset as an image, the resulting image displayed will not have a smooth color scale, and it may look like the palette is recycled midway through.

A patch is available for HDFView to correct this problem. You can download the jar files containing the fix to the problem. Only the jhdfview.jar file is strictly necessary to correct this problem. See the README for other problems fixed with these jar files.

High Level Packet Table APIs and Variable Length Datatypes

Variable length Packet Tables are not supported. There are no tests or examples for using variable length fields in Packet Tables. Please use the HDF5 APIs to work with Variable Length datatypes.

The Linux 32-bit version of HDF-Java 2.5 fails on a Linux 64-bit machine

This problem is known to occur on a machine with gcc 3.4 on it, but may also occur with other versions of gcc. HDF-Java 2.5 was built with gcc 4.2. If you encounter this problem, try upgrading the version of gcc on your machine to gcc 4.2.

The problem also occurs with HDF5.

Building HDF5 1.8.2 w/gfortran 4.3, make fails with undefined reference to__h5global_MOD*

HDF5 1.8.2 was tested with gfortran 4.2. There is a known compiler bug in gfortran 4.3, which causes the HDF5 build to fail with "undefined reference to__h5global_MOD*" errors. This bug has been fixed. See:

   gcc bugzilla #38171

   [Bug fortran/38171] [regression] equivalence and nested modules broken

Building HDF5 1.8.2, error: call to __open_missing_mode declared with attribute error: open with O_CREAT in second argument needs 3 arguments

This problem has been fixed for the next release. A workaround is to edit the ./perform/zip_perf.c file in the source and change line 552 to:

    output = open(filename, O_RDWR | O_CREAT, S_IRWXU);

Problems Reported with HDF5 1.8.1

Tools tests fail when build HDF5 1.8.1 with "make check install"

There is a bug in HDF5 1.8.1 when building the library with "make check install". We will be fixing this bug for the next release.

The workaround is to run each step separately:

If the .out files look good, proceed to the next step.

Building HDF5, the Object Header Test, "Testing message deletion", fails with the error "H5O_mtime_decode(): badly formatted".

This problem is due to an incorrect timestamp, and it will be fixed in HDF5 1.8.2. To ignore this test for now, see this Frequently Asked Question. You can also temporarily change the timezone to resolve the problem.

Here are details on the problem:

This problem can be reproduced by setting the TZ variable to "EET-2EEST", which is the timezone for Eastern Europe. The timestamp which causes this error is encoded as "19700101003321", which represents Jan. 1, 1970 00:33:21. When translating this to UTC time from mktime, 2 hours are subtracted, thus putting it pre-Epoch.

Problems Reported in HDF5 1.6.7

"make check" fails when testing h5dump with the error, "...tail: cannot open '+41'"

This indicates that your system is a new platform that is not supported by The HDF Group yet. It uses the newer Gnu Coreutil syntax. The "tail -n +4" syntax is the newer POSIX syntax that is not recognized by systems that conform to the older POSIX syntax. Changing the command to use this new syntax will not work on some of the current platforms (e.g., SunOS, ...) that we support. Therefore, this change has not been added to the software.

Run the HDF5 "make check" by first setting the environment variable, _POSIX2_VERSION to the value "199209". For example,

setenv _POSIX2_VERSION 199209

This solution is documented in the Gnu coreutils manual page:

   http://www.gnu.org/software/coreutils/manual/coreutils.txt

Problems Reported in HDF5 1.6.6

Solaris and gcc

HDF5 was not built and tested with the GNU compilers, and users have reported problems. One user was able to build on Solaris by removing the -std=c99 option from CFLAGS in all of the Makefiles.

tar on Solaris

If you encounter a problem untarring the HDF5 distribution using Sun's tar, then use GNU tar.

Problem Reported in HDF5 1.6.2(3)

Building 5-1.6.2(3) using Intel 8.1, get ULLONG_MAX error

When building HDF5-1.6.2 (or 5-1.6.3) using Intel 8.1, the make fails with the error:

   ...src/H5S.c(842): error: identifier "ULLONG_MAX" is undefined

The problem is resolved by adding the -c99 option when building. With 5-1.6.4, we added this option to the build so the problem no longer occurs.

This error does not occur with versions of Intel prior to 8.1.

- - Last modified:January 27th 2012