The HDF Group at the ESIP Winter Meeting
Ted Habermann, The HDF Group
The first week of January brought leaders of the Earth Science informatics community together in Washington DC for the winter meeting of the Federation of Earth Science Information Partners (ESIP Federation). This year NOAA’s Environmental Data Management Workshop was co-located with ESIP for five exciting days of data management ideas and discussions. The HDF Group was well represented at both of these meetings with presentations on metadata and data containers.
The ESIP Federation comes together twice each year to discuss topics around changing technology, data, information and knowledge in support of society. ESIP meetings are interdisciplinary and inclusive. Among the attendees are Earth science data and information technology practitioners; researchers representing a variety of scientific domains that include land, atmosphere, ocean, solid earth, ecology, data and social sciences; science educators; and anyone working in science and technology-related fields who is interested in advancing Earth science information best practices in an open and transparent fashion.
We started on Tuesday when I suggested that we need to think about making connections between documentation for collections of related files and the documentation that exists in the files themselves, a continuation of some of the ideas I introduced during the “Granules Are Forever” keynote talk during the 2012 HDF workshop. I proposed a new approach to implementing standard ISO metadata in HDF files. I expect that this will be a topic for the ESIP Documentation Cluster over the next year.
The metadata theme continued into the ESIP meeting with a talk by The HDF Group’s John Kozimor on the metadata evaluation and improvement work he is doing with the U.S. Big Earth Data Initiative. John’s comments sparked vigorous discussion with his analysis of NASA Climate Data Initiative metadata, Catalog Services for the Web and DataCite. Representatives of the Research Data Alliance and the U.S. Geological Survey drove questions about comparisons between various community recommendations and CSDGM metadata standards. I followed up with a detailed analysis of people and organizations in the NASA Common Metadata Repository and a proposal that ISO Components might be useful for improving the completeness and consistency of this information. The metadata discussions continued into the poster session with posters on Arctic metadata management and metadata evaluation and improvement.
The HDF Group contributions continued on Friday with a session on the Data Container study that HDF is participating in as part of NASA’s Advanced Information System Technology Program (AIST). Presentations by Chris Mattmann and Ed Armstrong from NASA’s Jet Propulsion Lab and Phil Yang from George Mason University led off the session. Then came Aleksandar Jelenak and me from The HDF Group. Together these presentations began to lay a foundation for understanding how various data containers (e.g. AsterixDB, Rasdaman, and HDF) can be used to facilitate different analytical use cases. The presentations focused on mechanics of data preparation and migration between containers. I described the role of the HDF Virtual Object Layer (VOL) in maintaining stable APIs while data migrates across containers.
In addition to the general data management discussions, members of The HDF Group participated in leadership discussions for the National Science Foundation’s EarthCube project (Lindsay Powers) and the ESIP members that develop Earth Science applications (Ted).