Webinar Followup: Parallel I/O with HDF5 and Performance Tuning Techniques
On June 26, 2020, The HDF Group employee, Scot Breitenfeld presented a webinar called “Parallel I/O with HDF5 and Performance Tuning Techniques.”
Webinar Announcement: Parallel I/O with HDF5 and Performance Tuning Techniques
Scheduled for June 26, 2020 11:00 a.m. CDT, this webinar, presented by Scot Breitenfeld is designed for users who have had exposure to HDF5 and MPI I/O and would like to learn about doing parallel I/O with the HDF5 library.
Handling (and ingesting) data streams at 500K mess/s
By Francesc Alted. He is a freelance consultant and developing author of different open source libraries like PyTables, Blosc, bcolz and numexpr and an experienced programmer in Python and C. Francesc collaborates regularly with the The HDF Group in different projects. We explain our solution for handling big data streams using HDF5 (with a little help
HDF5 Data Compression Demystified #2: Performance Tuning
Internal compression is one of several powerful HDF5 features that distinguish HDF5 from other binary formats and make it very attractive for storing and organizing data. Internal HDF5 compression saves storage space and I/O bandwidth and allows efficient partial access to data. Chunk storage must be used when HDF5 compression is enabled.
Parallel and Large-scale Simulation Enhancements to CGNS
The current improvement of using collective I/O to reduce the number of independent processes accessing the file system helped to improve the metadata reads for cgp_open substantially, yielding 100-1000 times faster execution times over the previous implementation.
HDF at the 2015 Oil & Gas High Performance Computing Workshop
The workshop program has two main tracks, one on HPC-oriented technologies that support the industry, and one on oil & gas technologies and how they can leverage HPC.