A Kind of Magic: Storing Computations in HDF5

The purpose of this introduction is to highlight and celebrate a community contribution the impact of which we are just beginning to understand. Its principal author, Mr. Lucas C. Villa Real, calls it HDF5-UDF and describes it as “a mechanism to generate HDF5 dataset values on-the-fly using user-defined functions (UDFs).” This matter- of-fact characterization is quite accurate, but I would like to provide some context for what this means for us users of HDF5.

Can We Remove the Autotools?

HDF5 can be built using two build systems: the Autotools (since HDF5 1.0) and CMake (since HDF5 1.8.5). For a long time, the Autotools were better maintained and CMake was more of an “alternative” build system that we primarily used for handling Windows support (the legacy Visual Studio projects were removed in HDF5 1.8.11).

This is no longer the case though—CMake support in HDF5 is (almost) as good as Autotools support and CMake, in general, is much more commonly used now than when we first introduced it.

So why are we still hanging on to the legacy Autotools?

Guided metadata improvement to increase data sharing and understanding

Lindsay Powers, The HDF Group The HDF Group is collaborating with the University of California, Santa Barbara and Data Observation Network for Earth (DataONE), to help scientific research communities enhance the consistency and quality of their metadata, to foster discovery, access and understanding of data resources. As part of this collaboration, on February 9, 2016,

The HDF Group appoints Neil Fortner as Chief HDF5 Software Architect

We are excited to announce the appointment of Neil Fortner as the new Chief HDF5 Software Architect. Neil has worked for The HDF Group as a software engineer since 2008. While at The HDF Group, he focused his talents in storage and HPC on improving performance, expanding the features, and improving the maintainability of the

Matthew Larson joins The HDF Group

Matthew Larson has joined The HDF Group as a software developer. Matthew is a recent graduate from the University of Illinois at Urbana-Champaign with a BS in Computer Science. While in school, Matthew held several internships where he worked in machine learning using AWS resources and data collection and served as a course associate for

Department of Energy Awards Grant to The HDF Group and Collaborators for Fusion Energy Data Management Tools

The HDF Group has been selected to receive a Department of Energy grant to develop a platform where data from different fusion devices is managed according to Findable, Interoperable, Accessible, and Reusable (FAIR) standards and UNESCO’s Open Science recommendations. The data will also be adapted for use with machine learning (ML) tools. Led by researchers at MIT, this collaborative project also includes Auburn University, William & Mary, and the University of Wisconsin-Madison.

Scroll to Top