HDF5 Build System: A Detailed Comparison of CMake and Autotools and why you’ll find a big change in HDF5 2.0

When working with a foundational library like HDF5, what happens behind the scenes matters. The build system—the collection of scripts that compile source code into libraries and tools—plays an essential role in your workflow. A reliable and consistent build system is key to a smooth experience. However, challenges can arise when a project supports two different systems. That’s why we talked to the community and removed Autotools from our upcoming release of HDF5 2.0.

To provide the best possible experience for developers, before we removed Autotools, we recently conducted an in-depth audit comparing HDF5’s modern CMake build system with the established Autotools. We tested both systems on Linux, enabling a wide range of features to identify where they align and diverge. The focus of this audit was not on new features but on ensuring fundamental aspects were solid. Let’s explore the findings!

The Great Flag Debate: Auditing Compiler Flags 🚩

Compiler flags are special instructions given to the compiler that influence how it builds software. For a library like HDF5, the flags used during a debug build are particularly critical, as they can mean the difference between a quick fix and hours of frustration.

The audit revealed some noteworthy discrepancies:

• Flags Missing from CMake: The Autotools build included several flags that were not present by default in a CMake debug build. For instance, flags like -fdiagnostics-urls=never and -fno-diagnostics-color enhance compiler output by improving readability but are toggled off during debug builds in CMake. Conversely, -fno-omit-frame-pointer is crucial for effective debugging, helping tools trace program execution. While this flag is on by default in Autotools, it’s off in CMake unless tied to the HDF5_ENABLE_SYMBOLS option. The audit suggests it should likely be enabled by default for debug builds.
• Hidden Debugging APIs: Autotools enables a suite of internal debugging macros (such as DH5AC_DEBUG, DH5D_DEBUG, etc.) that are invaluable for HDF5 developers. In contrast, these macros in CMake are controlled by the HDF5_ENABLE_DEBUG_APIS option, which is off by default unless performing a specialized “developer build.”
• A Ghost Flag: The audit found that H5_DEBUG_API is unused in the source code, indicating that the CMake logic that adds it can be safely removed.

Where Did Everything Go? The Installation Directory 📂

A clean and predictable file layout post-installation is vital for users and downstream packaging tools. The audit uncovered several key differences between the directory structures created by the two systems:

• bin/: CMake generates both h5cc/h5pcc and h5fc/h5pfc, whereas Autotools only creates one pair.
• include/: The CMake installation includes additional files for new development, such as Fortran headers like H5f90.h, which are missing from the Autotools build.
• lib/: This area is critical for compatibility.
• Library Naming: The HL Fortran libraries have different names; Autotools creates libhdf5hl_fortran.so, while CMake produces libhdf5_hl_fortran.so (with an underscore). To maintain backward compatibility, CMake will likely need to add a symbolic link with the old name.
• New Libraries: CMake introduces new Fortran stub libraries (libhdf5_f90cstub.a) and installs tools and Java libraries, which are not part of the Autotools build.
• Settings Files: The important libhdf5.settings file, which records build configurations, differs significantly. The CMake version often lacked information or had empty fields, which should be addressed.

A Conditional Conundrum: The #ifdef Audit

Preprocessor directives like #ifdef determine which parts of code are compiled, often based on the operating system or enabled features. Fortunately, the audit revealed no major features enabled by Autotools that were completely absent in CMake.

However, the audit identified some areas for cleanup and improvement:

• C11 Threads: A check for H5_HAVE_THREADS_H in CMakeLists.txt needs to be uncommented, as HDF5 now officially requires the C11 standard, which includes native threading support.
• Unused Checks: The code contains checks for features like H5_HAVE_PTHREAD_BARRIER and DAOS_SUPPORT that are never enabled by the build system. These represent “dead code” paths that could be eliminated.

The Path Forward

This detailed comparison highlights the minor yet significant differences between the two build systems. While CMake is a robust and effective system for HDF5, this audit provides a clear roadmap for improvement. By standardizing compiler flags, ensuring backward compatibility in library naming, and cleaning up unnecessary code, with the release of HDF5 2.0 we can further enhance the HDF5 build experience for the entire community.