Blog

The HDF Group – who we are

We thought it would be good to kick off the HDF Blog series with a short explanation of who we are and why we exist.

The HDF Group started in 1987 at the National Center for Supercomputing Applications (NCSA) at the University of Illinois in Urbana, Illinois. Here’s an email from the first meeting of the group:

The first version of HDF was implemented the following spring. Over the next 10 years HDF enjoyed widespread interest and adoption for managing scientific and engineering data. The NASA Earth Observing System (EOS) was an early adopter of HDF. NASA provided much of the funding and technical requirements that made HDF a robust technology, able to support mission-critical applications.

By 1996 it became clear that HDF was not going to adequately address the demands of the next generation of data volumes and computing systems, and in 1998 a second version, called HDF5, was implemented. HDF5 was more scalable than the original HDF (now called HDF4), and had many other improvements. The Department of Energy’s Sandia, Los Alamos, and Lawrence Livermore National Laboratories provided the core funding, technical requirements, and many of the people that made the new format possible. HDF5 quickly replaced HDF4 in popularity, and spread even more rapidly.

In the late 1990s and early 2000s the HDF Group faced increasing demands to ensure that HDF was robust, that HDF5 kept up with advancing technologies and data demands, and that we offer high quality professional support for HDF users. It soon became clear that the HDF Group could best serve these demands by striking out on its own, as an entity separate from the University and NCSA, who had nurtured us so well for 18 years.

In January 2005, The HDF Group was incorporated as a not-for-profit company. In July 2006, twelve of us set up shop in the University of Illinois Research Park, and we got ourselves a logo:

 

Our initial funding came from a financial company that had adopted HDF5 to help gather and manage multiple high speed, high volume market data feeds. We provided them with support and a number of new capabilities in HDF5. The NASA EOS soon joined with contracts for the new company, as did two of the three DOE Labs.

The HDF Group chose to be a non-profit because we had a public mission, and we wanted to feel confident that the company would not be diverted from that mission for reasons of financial gain.

The HDF Group’s mission is:

To provide high quality software for managing large complex data, to provide outstanding services for users of these technologies, and to insure effective management of data throughout the data life cycle.

The mission has two goals:

1. To create, maintain, and evolve software and services that enable society to manage large complex data at every stage of the data life cycle.
2. To establish and maintain a sustainable organization with a highly-skilled and committed team devoted to accomplishing the first goal.

The rest is details. We’ll be getting into those details in future blog posts, and we’re hoping some of you will contribute.

Meanwhile, send your comments and questions. We’d love to hear from you.  Subscribe to our blog posts on the sidebar.  And if you’d like to do a post, please send an email to blog@hdfgroup.org.

Mike Folk