mobile-logo

Archive

Why HDF® Technologies? Physics, the branch of science that deals with the nature and properties of matter and energy, spans a wide variety of fields, including astronomy, astrophysics, geophysics, weather and climate, biophysics, material science, and many more. Physics was probably the first branch of science to use computers in a significant way. Today, both computational and experimental physics of every type rely completely on computers and their ability to handle massive amounts of complex data. HDF is the option of choice in many branches of physics because it offers so many different capabilities needed to manage physics data. It can effectively handle data of virtually any size, as exemplified by the trillion particle plasma physics simulation, which needed to store HDF5...

June 27, 2017

Why HDF® Technologies? Medical imagery Electron microscopy, functional MRI, biomedical simulations, particle detectors and other instruments generate increasingly high resolution imagery at unprecedented rates. The metadata associated with this imagery can be very complex, and is usually as important to understanding the data as the images themselves. More often than not, the processes that work with these images need to be able to search the images quickly and access specific parts of the images, hopefully without the burden of having to uncompress large amounts of data or perform exhaustive searches. Traditional formats for storing this data, such as TIFF and DICOM are either difficult to scale, or unable to perform in terms of scalability, access speeds, and the ability to accommodate large and...

June 27, 2017

Why HDF® Technologies? Each new generation of sequencing instruments generates orders of magnitude more data than the previous generation, at a fraction of the cost. Those who must acquire, process, analyze, and archive that data are continually scrambling to adapt their data management tools to this explosive growth. Downstream processing requires that the data be available in the right form to a variety of analytical methods, which historically has meant reformatting the data many times, as well as generating new kinds of data, as it passes through the workflow. The ability of HDF5 to accommodate virtually every kind of data in a single file has been recognized by many researchers as providing the ideal container for sequencing data. Sequences, images, SNP matrices,...

June 27, 2017

Why HDF® Technologies? HDF5 has broad adoption in the financial services industry including High-frequency trading (HFT) firms, hedge funds, investment banks, pension boards and data syndicators. All sizes and types of financial firms rely on massive amounts of data for trading, risk analysis, customer portfolio analysis, historical market research, and many other data intensive functions. Many HDF adopters in finance have very large datasets, very fast access requirements, or very complex datasets. Others turn to HDF because it allows them to easily share data across a wide variety of computational platforms using applications written in different programming languages. Some use HDF to take advantage of the many open-source and commercial tools that understand HDF. In this environment more and more financial firms, research...

June 27, 2017

Why HDF® Technologies? NASA’s Earth Observing System (EOS) consists of satellites whose observations include the land surface, atmosphere, oceans, and snow and ice. The first EOS satellite, Terra, was launched in 1999, and produced about a terabyte of data per day. By 2015, 18 EOS satellites were producing 16 TB per day stored in 12 data archive centers that specialized in different aspects of the Earth system, such as snow and ice, physical oceanography, severe weather, land cover, and global precipitation. Now, totaling over 14 petabytes, the data encompasses 9,400 different data products. In 2015, 32 terabytes per day were distributed to more than 2.4 million and users worldwide. In the mid-1990s EOS chose HDF as its data format, and HDF, in...

June 27, 2017

Why HDF® Technologies? In today’s world, every aspect of engineering must handle data for design, building, testing, monitoring and use. Moreover, since engineering generally concerns itself with quality and reliability, it is important that engineering data be complete and accurate, and that engineering applications be able to perform reliably at whatever speed the environment demands. It is this comprehensive set of requirements that makes HDF5 especially attractive for many fields of engineering. Automotive and aerospace engineering applications provide excellent examples of the value of HDF5 for engineering applications. These fields require data acquisition and management solutions that can handle multiple streams of sensor data as well as a variety of metadata. They need to be able to stream data very quickly into a...

June 27, 2017

Why HDF® Technologies? An astounding variety of physical phenomena involve fluid flow including breathing and drinking, heating and ventilation, flying and sailing, engine combustion and galaxy formation. Computational fluid dynamics (CFD) uses numerical methods and algorithms to solve and analyze problems that involve fluid flows. CFD, high-performance computing and big data all go hand-in-hand. For CFD researchers and engineers, it often seems that there is never enough computing resources available to solve the problems at hand. On the one hand, every order of magnitude increase in computing power leads to the ability to solve new CFD problems. However, the same increases in computing power lead to the creation of even more data delivered at faster speeds, furthering the need for even more...

June 27, 2017

Why HDF® Technologies? An amazingly broad range of astronomy instruments, such as optical and radio telescopes and the gravitational wave laser interferometer, are collecting unprecedented and unmanageable amounts of data about the very largest in the very smallest objects and phenomena in our universe. Much of this data is so voluminous and arrives at such great speeds that it can only be archived for short periods of time. What makes HDF5 especially attractive in the world of astronomy is the combination of HDF5’s ability to handle data of any size and fast acquisition rates, to combine and effectively integrate data from many different sources, to run on virtually any computing platform from laptops to petascale computers, and to provide a software ecosystem...

June 27, 2017

Why HDF Technologies? HDF5 is the ideal library and format to manage the enormous amounts of data produced by very fast parallel i/o processes used to conduct simulations When applications are particularly data and compute intensive, engineers and researchers often turn to HDF5 to manage their storage and I/O requirements, and at the same time have the data in a format that’s well understood by by important applications in the field, such as ANSYS and Matlab. Semiconductor wafer testing uses probes to detect flaws in the fabrication process. Huge amounts of data can be generated in this process, resulting in tables that are far larger than typical RDBMS can handle efficiently so they turn to HDF5 because HDF5 can store objects of any...

June 27, 2017

Why HDF Technologies? HDF5 can accommodate virtually every kind and size of data, making it an ideal container for data used in oil exploration. HDF5 software was built for performance and scalability throughout the data life cycle and on every type and size of computing system, making it a perfect vehicle for managing industry data through complex workflows and for long term storage. The HDF5 software ecosystem includes all major programming languages and operating systems, and a wide array of popular analysis and visualization packages used by the industry, making it easy to work with for everyone from software developers to field engineers to research scientists.   How HDF Technologies are Used Analysis, storage, and sharing of enormous amounts of complex and varied data including...

June 27, 2017
Next Page »
  • 1