mobile-logo

Uncategorized

Scot Martin, Harvard University, HDF Guest Blogger HDF5 storage is really interesting. To me, its format has no fixed structure, but instead is based on introspection and discovery. Seems great to me; Mathematica has its origins first in artificial intelligence, so we ought to be able to do something here.  Approaching twenty-two years with Mathematica and almost a “Hello, World!” ability in C, I decided to jump right in. Enter The HDF Group's P/Invoke for my salvation. Here’s how we make use of it in Mathematica: LoadNETAssembly["HDF.PInvoke.dll"] Bang! Ready to go in Mathematica. Here’s a proof of concept for how it works: Module[ (* The three symbols should have initial values so that there is *) (* memory allocation when Mathematica interfaces with P/Invoke. *) {major=0,minor=0,revision=0,return}, CompoundExpression[ (* access...

May 5, 2017

Christian Hoene, Symonics GmbH; and Piotr Majdak, Acoustics Research Institute; HDF Guest Bloggers Spatial audio - 3D sound.  Back in the ‘70’s, “dummy head” microphones were used to create spatial audio recordings. With headphones, one was able to listen to those recordings and marvel at the impressive spatial distribution of sounds – just like in real life. [caption id="attachment_11132" align="aligncenter" width="624"] Displays the difference between listening to a real source and listening to realistic virtual sounds via headphones[/caption] Nowadays, we have a much better understanding of the human binaural perception and we can even simulate spatial audio signals with the help of computers.  Indeed, a modern virtual reality (VR) headset such as the Oculus Rift or Samsung Gear utilizes 3D audio to allow...

April 26, 2017

The HDF Server allows producers of complex datasets to share their results with a wide audience base. We used it to develop the Global Fire Emissions Database (GFED) Analysis Tool, a website which guides the user through our dataset. A simple webmap interface allows users to select an area of interest and produce data visualization charts. ...

April 18, 2017

Mark Miller, Lawrence Livermore National Laboratory, Guest Blogger The HDF5 library has supported the I/O requirements of HPC codes at Lawrence Livermore National Labs (LLNL) since the late 90’s. In particular, HDF5 used in the Multiple Independent File (MIF) parallel I/O paradigm has supported LLNL code’s scalable I/O requirements and has recently been gainfully used at scales as large as 1,000,000 parallel tasks. What is the MIF Parallel I/O Paradigm? In the MIF paradigm, a computational object (an array, a mesh, etc.) is decomposed into pieces and distributed, perhaps unevenly, over parallel tasks. For I/O, the tasks are organized into groups and each group writes one file using round-robin exclusive access for the tasks in the group. Writes within groups are serialized but...

March 21, 2017

Release of HDF5-1.8.18 (Newsletter #152) - 11/16/16 Release of HDF Java Products for HDF5-1.8 (HDFView 2.13, HDF JNI 3.2.1) (Newsletter #151) - 7/25/16 Release of HDF 4.2.12 (Newsletter #150) - 6/30/16 Release of HDF5-1.10.0-patch (Bulletin) - 5/26/16 Release of HDF5-1.8.17 (Newsletter #149) - 5/13/16 Release of HDF5-1.10.0 (Newsletter #148) - 3/31/16 ...

November 17, 2016

David Pearah, The HDF Group Hello again, HDF User Community, As I mentioned in my last blog post -- HDF: The Next 30 Years (Part 2) -- we're looking for ways to better engage our users, which includes providing better tools for you to get support from the HDF Community.  We are looking for your input on three things: the HDF User Forum, Roadmap for HDF5, and Focus Groups - please take a few minutes to complete these short surveys and let us know what you think! Discussion Board and Listserv Forum Feedback:  LINK  We currently provide a listserv as the primary way that folks post questions and feedback to the community, and we would like your feedback since we're looking at web-based tools to complement or...

November 3, 2016

The HDF Group’s HDF Server has been nominated for Best Use of HPC in the Cloud, and Best HPC Software Product or Technology in HPCWire’s 2016  Readers’ Choice Awards. HDF Server is a Python-based web service that enables full read/write web access to HDF data – it can be used to send and receive HDF5 data using an HTTP-based REST interface. While HDF5 provides powerful scalability and speed for complex datasets of all sizes, many HDF5 data sets used in HPC environments are extremely large and cannot easily be downloaded or moved across the internet to access data on an as-needed basis.  Users often only need to access a small subset of the data.  Using HDF Server, data can be kept in one...

September 16, 2016

David Pearah, The HDF Group Hello HDF Community! Thanks for the warm welcome into the HDF family: in my 4+ months as the new CEO, I've been blown away by your passion, diversity of interests and applications, and willingness to provide feedback on:  1. why you use HDF5?, and  2. how can HDF5 be improved? I also want to thank my predecessor Mike Folk for his invaluable and ongoing support. The HDF community is growing fast: when I last checked, there are nearly 700 HDF5 projects in GitHub! I've had the privilege of connecting via phone/web with dozens of you over the past few months. Across all of my discussions, one piece of feedback came back loud and clear: The HDF Group needs to be more engaged with its users and help foster...

August 17, 2016

DOE has continued to partner with The HDF Group, supporting development of HDF5 through two generations of computing; sponsoring this development has benefited the entire HDF5 user community. Today, DOE supports current HDF5 R&D to ensure that the data challenges of third generation exascale computing ...

August 17, 2016
« Previous Page — Next Page »