Blog

This poster from Aleksandar Jelenak and Dana Robinson of The HDF Group runs through several strategies to optimize HDF5 and netCDF-4 files for the cloud, including consolidating internal metadata, setting a large chunk size, and avoiding or minimizing the use of variable length datatypes. Several code examples for each situation are included. You're going to want to view the full size PDF....

The HDF Group has been selected to receive a Department of Energy grant to develop a platform where data from different fusion devices is managed according to Findable, Interoperable, Accessible, and Reusable (FAIR) standards and UNESCO’s Open Science recommendations. The data will also be adapted for use with machine learning (ML) tools. Led by researchers at MIT, this collaborative project also includes Auburn University, William & Mary, and the University of Wisconsin-Madison....