HDF5 Resources for ECP Users

On Friday, April 30, The HDF Group's Gerd Heber presented HDF5 Application Tuning: There is more than one way to skin a cat(fish), Part 2. Before returning to application tuning (in part 3), in this second part of the series, we take a closer look at HDF5 performance variability. We highlight the main variability sources, their impact on performance, and considerations for HDF5 container design. https://youtu.be/JEnYLWpBjso...

Greetings! We have scheduled a new webinar for Friday, April 30 at 11:00 a.m. CDT. HDF5 Application Tuning: There is more than one way to skin a cat(fish), Part 2 Abstract: Before returning to application tuning (in part 3), in this second part of the series, we take a closer look at HDF5 performance variability. We highlight the main variability sources, their impact on performance, and considerations for HDF5 container design. Please register to join us live. This event will be recording and shared with the community....

On February 12, 2021, we were pleased to host Lucas Villa Real of IBM Research to discuss his project HDF5-UDF, a data virtualization tool for HDF5. The tool enables users to associate logic in source code form (i.e., in user-defined functions, written in Python, C/C++, or Lua) with HDF5 datasets. Such UDFs are compiled into a binary form (which often takes no more than a few KB) and embedded into HDF5; once an application reads such a dataset, HDF5-UDF executes that binary code and generates the data on-the-fly. Lucas has just released HDF5-UDF 1.2 which offers several new features: among other benefits, it makes it possible to easily virtualize CSV files so they look like regular HDF5 datasets. Attached you'll find the slide deck...

The HDF Group’s Gerd Heber hosts a weekly session where he tries to answer attendee questions and, for example, go over the previous week’s HDF Forum posts. The HDF Clinics are free sessions intended to help users tackle real-world HDF problems from a common cold to severe headaches and offer relief where that’s possible. As time permits, we will include how-tos, offer advice on tool usage, review your code samples, teach you survival in the documentation jungle, and discuss what’s new or just around the corner in the land of HDF. Please submit questions/topics in this google doc. One-time registration required. Thanks to all who attended our first HDF Clinic on February 9, 2021. The resources from this clinic are archived here. Gerd's Notes https://youtu.be/g5h_YlvI9Aw  ...

On Friday, July 31, The HDF Group employee Chen Wang presented a Study of HACC-IO Benchmarks. We wanted to share these presentations with our community. Slide Deck https://youtu.be/AkcOwg3qLZo   Additionally, Chen Wang also wrote the white paper, An I/O Study of ECP Applications....