Tobias Weinzierl, Durham University, UK, Sven Köppel, FIAS, Germany, Michael Bader, TUM, Germany, HDF Guest Bloggers ExaHyPE develops a solver engine for hyperbolic differential equations solved on adaptive Cartesian meshes. It supports various HDF5 output formats. Exascale computing is expected to allow scientists and engineers to simulate, and ultimately understand, wave phenomena with unprecedented accuracy […]
Mark Miller, Lawrence Livermore National Laboratory, Guest Blogger The HDF5 library has supported the I/O requirements of HPC codes at Lawrence Livermore National Labs (LLNL) since the late 90’s. In particular, HDF5 used in the Multiple Independent File (MIF) parallel I/O paradigm has supported LLNL code’s scalable I/O requirements and has recently been gainfully used
By Francesc Alted. He is a freelance consultant and developing author of different open source libraries like PyTables, Blosc, bcolz and numexpr and an experienced programmer in Python and C. Francesc collaborates regularly with the The HDF Group in different projects. We explain our solution for handling big data streams using HDF5 (with a little help
A few years ago, I was looking for a data format with low latency block and stream support. While protocol buffers offered streams, it was lacking indexed block access. Soon, I realized I was looking for a container with file system-like properties. When I examined HDF5, I found it was very close to what I needed to store massive financial engineering datasets….
We are pleased to post this white paper from The HDF Group intern, Chen Wang. This paper looks at the steps of analyzing and tuning the HACC-IO benchmarks, the impact of different access patterns, stripe settings and HDF5 metadata. It also compares the five benchmarks on two different parallel file systems, Lustre and GPFS and
The HDF Group’s technical mission is to provide rapid, easy and permanent access to complex data. FishEye’s vision is “Synthesizing the world’s real-time data”. This white paper is intended for embedded system users, software engineers, integrators, and testers that use or want to use HDF5 to access, collect, use and analyze machine data. FishEye has developed an innovative process that provides the most efficient method to expose data from embedded systems that simplifies and liberates data for real-time analysis, machine learning, and cloud-enabled services.
Hermes 0.9.8 has been released. This release features tagging. Tags enable users to semantically define associations between blobs and provide an intuitive way of locating blobs which are related.
The HDF5 1.14.6 release is now available from the HDF5 1.14.6 download page. This is a maintenance release with some minor new changes: Reverted a change to the library’s handling of UTF-8 file names. A change was made in the HDF5 1.14.4 release to address some issues with the library’s handling of code pages and
by John Readey “For Heaven’s sake, do not confound HDF5 with anything else!” —Friedrich Nietzsche, Ecce Homo Alan Turing devised the Turing test (he referred to it as the “Imitation Game”) in 1950 to determine whether a machine could think. He believed that if machines could give answers that were indistinguishable from human answers, then
I am pleased to announce an important change in our leadership at The HDF Group. M. Scot Breitenfeld has been appointed our new Director of Engineering. Since joining our organization in 2008, Scot has contributed significantly to our mission and has developed an extensive understanding of our technology and community over the past 17 years.
