Innovation

The purpose of this introduction is to highlight and celebrate a community contribution the impact of which we are just beginning to understand. Its principal author, Mr. Lucas C. Villa Real, calls it HDF5-UDF and describes it as "a mechanism to generate HDF5 dataset values on-the-fly using user-defined functions (UDFs)." This matter- of-fact characterization is quite accurate, but I would like to provide some context for what this means for us users of HDF5....

The HDF Group's formal comment to a DOE request on stewardship for scientific and high-performance computing was published in the Federal Register. We had a joint position paper at January's ASCR Workshop on Visualization for Scientific Discovery, Decision-Making, & Communication. The paper is called Whither Visualization Logic and was written by Leigh Orf (University of Wisconsin), Lucas Villa Real (IBM Research), and Gerd Heber (The HDF Group). Thomas Caswell (active h5py contributor, Brookhaven National Laboratory) also has a position paper called Visualization of Structured Data. Additionally, we have two position papers at the ASCR Workshop on the Management and Storage of Scientific Data. The first was led by Jerome Soumagne (The HDF Group), The Twilight of I/O as a User Concept and is joint work with Andres Marquez from the PNNL....

We are excited to announce a new strategy of delivering HDF5 features: Experimental Releases. Experimental releases allow us to get major new features into the hands of our users so that they can test the features and provide feedback before we integrate them into a subsequent maintenance release. Our first experimental release, HDF5 version 1.13.0, is now available....

[vc_row disable_margin_on_row="no"][vc_column][vc_column_text] By John Readey   [/vc_column_text][mkd_message type="normal"] HSDS (Highly Scalable Data Service) is available and fully functional on Azure. Try HSDS out on the Azure Marketplace today! [/mkd_message][vc_column_text] Abstract HSDS (Highly Scalable Data Service) is a cloud-native data service designed to fully support the HDF5 data model and API while taking advantage of cloud-based features such as object storage and on-demand compute. HSDS is an open-source, Apache2 licensed product which has been used by many organizations to provide access to HDF data over the web. Originally designed with Amazon's AWS as the target platform, The HDF Group has worked with Microsoft Azure to enable HSDS to run equally well on Azure. Enhancements made to HSDS include: support for Azure Blob Storage, authentication with Azure Active Directory,...

The HDF Group’s technical mission is to provide rapid, easy and permanent access to complex data. FishEye's vision is "Synthesizing the world’s real-time data". This white paper is intended for embedded system users, software engineers, integrators, and testers that use or want to use HDF5 to access, collect, use and analyze machine data. FishEye has developed an innovative process that provides the most efficient method to expose data from embedded systems that simplifies and liberates data for real-time analysis, machine learning, and cloud-enabled services....

HSDS (Highly Scalable Data Service) is a REST-based service for reading and writing HDF data. Initially developed as a NASA Access 2015 project, the HDF Group has continued to invest in the project, and as we'll see, the latest version has a bevy of new and interesting features....

On Friday, July 31, The HDF Group employee Chen Wang presented a Study of HACC-IO Benchmarks. We wanted to share these presentations with our community. Slide Deck https://youtu.be/AkcOwg3qLZo   Additionally, Chen Wang also wrote the white paper, An I/O Study of ECP Applications....