Innovation

On Wednesday, Nov 16, 2022 09:00 AM The HDF Group hosted a webinar on NeXpy. NeXpy is a GUI application designed to to facilitate creating, reading, visualizing, and manipulating data stored in HDF5 files. Although it was primarily designed to handle neutron and x-ray scattering stored using the NeXus format, most of its functionality is applicable to other types of scientific data stored in HDF5 files or even imported in a variety of formats....

HDF5 can be built using two build systems: the Autotools (since HDF5 1.0) and CMake (since HDF5 1.8.5). For a long time, the Autotools were better maintained and CMake was more of an "alternative" build system that we primarily used for handling Windows support (the legacy Visual Studio projects were removed in HDF5 1.8.11). This is no longer the case though—CMake support in HDF5 is (almost) as good as Autotools support and CMake, in general, is much more commonly used now than when we first introduced it. So why are we still hanging on to the legacy Autotools?...

Highly Scalable Data Service principal architect John Readey covers an update to the Highly Scalable Data Service. The max request size limit per HTTP request no longer applies with the latest HSDS update. In the new version large requests are streamed back to the client as the bytes are fetched from storage. Regardless of the size of the read request, the amount of memory used by the service is limited and clients will start to see bytes coming back while the server is still processing the tail chunks in the selection. The same applies for write operations—the service will fetch some bytes from the connection, update the storage, and fetch more bytes until the entire request is complete. Learn more about...

If you are looking to store HDF5 data in the cloud there are several different technologies that can be used and choosing between them can be somewhat confusing. In this post, I thought it would be helpful to cover some of the options with the hope of helping HDF users make the best decision for their deployment. Each project will have its own requirements and special considerations, so please take this as just a starting point....

The purpose of this introduction is to highlight and celebrate a community contribution the impact of which we are just beginning to understand. Its principal author, Mr. Lucas C. Villa Real, calls it HDF5-UDF and describes it as "a mechanism to generate HDF5 dataset values on-the-fly using user-defined functions (UDFs)." This matter- of-fact characterization is quite accurate, but I would like to provide some context for what this means for us users of HDF5....