Lindsay Powers, The HDF Group The 2015 HDF workshop held during the ESIP Summer Meeting was a great success thanks to more than 40 participants throughout the four sessions. The workshop was an excellent opportunity for us to interact with HDF community members to better understand their needs and introduce them to new technologies. You […]
Quincey Koziol, The HDF Group “A supercomputer is a device for turning compute-bound problems into I/O-bound problems.” – Ken Batcher, Prof. Emeritus, Kent State University. HDF5 began out of a collaboration between the National Center for Supercomputing Applications (NCSA) and the US Department of Energy’s Advanced Simulation and Computing Program (ASC), so high-performance computing (HPC)
David Dotson, doctoral student, Center for Biological Physics, Arizona State University; HDF Guest Blogger Recently I had the pleasure of meeting Anthony Scopatz for the first time at SciPy 2015, and we talked shop. I was interested in his opinions on MDSynthesis, a Python package our lab has designed to help manage the complexity of raw and derived
Mohamad Chaarawi, The HDF Group Second in a series: Parallel HDF5 In my previous blog post, I discussed the need for parallel I/O and a few paradigms for doing parallel I/O from applications. HDF5 is an I/O middleware library that supports (or will support in the near future) most of the I/O paradigms we talked
The current improvement of using collective I/O to reduce the number of independent processes accessing the file system helped to improve the metadata reads for cgp_open substantially, yielding 100-1000 times faster execution times over the previous implementation.
John Readey, The HDF Group Editor’s Note: Since this post was written in 2015, The HDF Group has developed HDF Cloud, a new product that addresses the challenges of adapting large scale array-based computing to the cloud and object storage while intelligently handling the full data management life cycle. If this is something that interests
HDF adds structure, context, and organization to the data to create information that can be shared…
Ted Habermann, The HDF Group “A strong foundation is being built for sharing data and information to create community knowledge and wisdom. This foundation includes HDF5 as the data layer with community conventions and ISO metadata facilitating use and understanding.” We have experienced so many monumental technological shifts during the last several decades that, like
HDF Group is hosting a one-day workshop at the upcoming Federation for Earth Science Information Partners (ESIP) Summer Meeting in Asilomar, CA on July 14th. Please join us to learn about new HDF tools, projects and perspectives. There will also be an HDF Town Hall meeting on Wednesday afternoon July 15th
John Readey, The HDF Group We’ve recently announced a new viewer application for HDF5 files: HDF Compass. In this blog post we’ll explore the motivations for providing this tool, review its features, and speculate a bit about future direction for Compass. HDF Compass is a desktop viewer application for HDF5 and other file formats.
