The HDF Cloud platform addresses the challenges of adapting large scale array-based computing to the cloud and object storage while intelligently handling the full data management life cycle. Organizations that have existing HDF5-based applications running POSIX file storage platforms can migrate their HDF5 data to a cloud provider running on object storage without needing to rewrite their application or sacrifice on performance or scalability. In addition, organizations are able to publish HDF5-based data to a cloud-based system enabling broad data access by users both internal to their organization and/or external users including the general public.
Map individual HDF5 objects (datasets, groups, chunks) as objects in Object Storage to store HDF5 content in S3:
Each chunk (heavy outlines) get persisted as a separate object:
The HDF Group is excited to launch HDF Cloud in 2018. We’re currently beta testing HDF Cloud and are looking for additional beta testers. If you’re interested in further information, senior developer John Readey put on this recorded webinar on HDF Cloud. If you have any questions, or just want to stay updated on this new product, let us know.
This material is based upon work supported by NASA under award Number NNX16AL91A. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Aeronautics and Space Administration.