HDF: The Next 30 Years (Part 2)

Dave Pearah, The HDF Group

In my previous post—HDF: The Next 30 Years (Part 1)—I outlined the challenges and opportunities facing The HDF Group as an open source company. In a nutshell:

  • Opportunity: large-scale adoption around the world in many different industries with great community-driven development (700+ projects in Github)
  • Challenge: sufficient profit from existing business (consulting) to sustainably extend and maintain the core HDF5 platform

The HDF Group is blessed with an amazingly talented + passionate + dedicated team of folks who care deeply about the HDF community, and we’re all working together to determine the best path forward to sustainability, i.e. the NEXT 30 years. We want to share some of the steps that we’re already taking, and — more importantly — we are providing a way for you to get directly involved. In fact, we’re counting on your feedback! (See below.)

1. Work in Progress

New Website (Launched + Evolving): If you’ve visited recently, you’ll notice that it’s been completely redesigned. Our old website did a poor job of 1. explaining how HDF5 is used, and 2. showcasing all of the products and services we provide… It also just looked dated and didn’t reflect the high-tech company that we most certainly are. Given the scale of all the content + design changes, we’ve broken this project into phases:

  • Marketing:… for folks new to HDF
  • Support:… for our experienced users, we’ve retained much of the old layout and content while we plan our next round of changes
  • Blog:… where you’ll hear about state-of-the-art technical insights, innovations from the HDF community of users, new product releases, and more
  • Registration + Discussion Board (coming soon): As part of the process of getting to know the HDF community, we need a reliable way of not only reaching folks, but also understanding where + how HDF is used. While the vast majority of our website will always be available without registration, we’ll soon start requiring registration to reach the Downloads page and our upcoming Discussion Board.

How does the website refresh support sustainability?: In addition to better promoting our existing products and services, we hope to get a more detailed understanding of how HDF is used, which in turn will drive further insights on our product and service roadmap.

New Support Packages (Launched): As you may have noticed on our new website, we have completely redesigned our Support Packages: We heard repeatedly from our users that while our Help Desk was appreciated, frankly, the quality of the code is so high that it rarely requires that type of assistance (e.g. “I’ve been using HDF5 for 10+ years without a single issue.”) However, we did get feedback that there are many other areas where HDF expertise would be helpful, e.g. training, documentation, tuning, advanced coaching, etc.

As a result, we’ve added a number of new services — including Help Desk — to our Support offerings and we’ve created 3 tiers of technical support to provide a cost-effective option for every type of HDF user.

How do these Support Packages support sustainability? Unlike our Consulting Services, which primarily pay for the software and metadata engineers on the project — these Support Packages directly fund the team that maintains and tests our HDF5 Core platform. In other words, Support Packages = high-quality, supported HDF Core.

ODBC Connector (Launching in 1 Week):
I’ll have a separate blog post on this product launch, but suffice it to say that we’re very excited about the possibilities and opportunities that this opens for HDF5 users, e.g. accessing HDF files from Excel, Tableau, Qlik, etc.

How do Products — like ODBC — support sustainability? Building premium (i.e. paid) products around a free core is a widely-accepted pathway for many open source companies, and provides additional value for our existing users. Similar to Support Packages, the revenue from Products directly funds the team that maintains the HDF5 Core Platform.

2. Voice of Customer: YOU!

We have a LOT of ideas of how The HDF Group can drive towards sustainability through new Products and Services, but we want to hear from you. The HDF Group exists because of all the dedication and support that you have shown us over many years, and we want to know how the HDF platform can help you succeed with your own projects + initiatives.

We’re creating 2 ways for you to connect directly with me and the engineering team:

2017 Roadmap Survey: LINK

We’re looking for feedback on a variety of ideas that we’ve brainstormed internally and in collaboration with key folks in the HDF Community over the past 6 months. We’re specifically looking at efforts that create premium (i.e. paid) services and products that build on our open source core.

Live Focus Groups: LINK

In addition to the survey, we’re offering 3 live + interactive meetings over the next month where you can provide unfiltered feedback on the following topics:

  • HDF5 in the Cloud: AWS/Google/Azure + Object Storage
  • HDF5 in Hadoop
  • HDF5 Core Roadmap: what additional features + functionality are you looking for?

Discussion Board and Listserv Forum Feedback: LINK

We’re looking for ways to better engage our users, which includes providing better tools for you to get support from the HDF Community. We currently provide a listserv as the primary way that folks post questions and feedback to the community, and we would like your feedback since we’re looking at web-based tools to complement or replace this email-centric approach.

In addition, you should always feel free to respond here in the forum or send me an email at

We’re excited about the opportunities ahead, but we depend on you to help us be successful. I’m looking forward to engaging in a deeper dialogue with you and driving forward in 2017.

Dave Pearah

No Comments

Leave a Comment