Technical Insights

On February 6, 2020, the members of the ECP ExaIO project, Elena Pourmal and Scot Breitenfeld (The HDF Group), Quincey Koziol (NERSC), and Suren Byna (LBNL), presented HDF5 Tutorial at the ECP Annual Meeting. We've posted the slide deck and Q&A for your convenience....

The HDF5 European workshop, co-organized with ESRF, and sponsored by OpenIO and Omnibond took place on September 17-18, 2019. This event covered the latest HDF5 developments, HDF5 use cases from science and industry, and HDF5 Applications and Tools. This post is an archive of the recorded presentations of this event. ...

Why do we use HDF5? We moved to HDF5 for our simulation data in 2016 from using our own proprietary file format. HDF5 had been on our radar for some time and we spent a couple of years investigating it and other file formats before deciding which we should switch to. HDF5 met all the criteria we had at the time. Amongst the criteria were: performance in speed and size, an accepted standard for scientific data, being open source, providing additional tools....

Question and answers from the HDF5 C++ Webinar on January 24th, 2019. Read the followup questions and answers from presentations on H5CPP from Steven Varga, h5cpp Wrapper from Martin Shetty and Eugen Wintersberger and Ntuple: Tabular Data in HDF5 with C++ from Chris Green and Marc Paterno. ...

Join The HDF Group for a webinar consisting of three short presentations from the HDF5 community to learn about different approaches and exciting work being done by the HDF5 C++ community members. Steven Varga, Martin Shetty and Eugen Wintersberger, and Marc Paterno and Chris Green will share their vision for and experiences using HDF5 with C++....

A few years ago, I was looking for a data format with low latency block and stream support. While protocol buffers offered streams, it was lacking indexed block access. Soon, I realized I was looking for a container with file system-like properties. When I examined HDF5, I found it was very close to what I needed to store massive financial engineering datasets...