mobile-logo

Innovation

The HDF Group has been selected to receive a Department of Energy grant to develop a platform where data from different fusion devices is managed according to Findable, Interoperable, Accessible, and Reusable (FAIR) standards and UNESCO’s Open Science recommendations. The data will also be adapted for use with machine learning (ML) tools. Led by researchers at MIT, this collaborative project also includes Auburn University, William & Mary, and the University of Wisconsin-Madison....

September 1, 2023

On Wednesday, Nov 16, 2022 09:00 AM The HDF Group hosted a webinar on NeXpy. NeXpy is a GUI application designed to to facilitate creating, reading, visualizing, and manipulating data stored in HDF5 files. Although it was primarily designed to handle neutron and x-ray scattering stored using the NeXus format, most of its functionality is applicable to other types of scientific data stored in HDF5 files or even imported in a variety of formats....

November 17, 2022

HDF5 can be built using two build systems: the Autotools (since HDF5 1.0) and CMake (since HDF5 1.8.5). For a long time, the Autotools were better maintained and CMake was more of an "alternative" build system that we primarily used for handling Windows support (the legacy Visual Studio projects were removed in HDF5 1.8.11). This is no longer the case though—CMake support in HDF5 is (almost) as good as Autotools support and CMake, in general, is much more commonly used now than when we first introduced it. So why are we still hanging on to the legacy Autotools?...

November 14, 2022

Highly Scalable Data Service principal architect John Readey covers an update to the Highly Scalable Data Service. The max request size limit per HTTP request no longer applies with the latest HSDS update. In the new version large requests are streamed back to the client as the bytes are fetched from storage. Regardless of the size of the read request, the amount of memory used by the service is limited and clients will start to see bytes coming back while the server is still processing the tail chunks in the selection. The same applies for write operations—the service will fetch some bytes from the connection, update the storage, and fetch more bytes until the entire request is complete. Learn more about...

August 22, 2022

If you are looking to store HDF5 data in the cloud there are several different technologies that can be used and choosing between them can be somewhat confusing. In this post, I thought it would be helpful to cover some of the options with the hope of helping HDF users make the best decision for their deployment. Each project will have its own requirements and special considerations, so please take this as just a starting point....

August 8, 2022
Next Page »