Data Cubes unlocking Earth observation potential
In December 2019, the CSIRO US team hosted a week of targeted meetings for Alex Held, Centre Director and Neil Sims, remote sensing research scientist to socialise the Data Cube project for California to potential collaborators. The objective of the week was to engage with key partners and share the potential benefits of the Data Cube technology in California.
What is a Data Cube?
Petabytes of decades-long Earth observation data are now available globally, and efficiently extracting usable information for decision making can be a challenge. Data cubes address this challenge by linking the highest quality image datasets to powerful computational and analytics systems.
The traditional approach for storing satellite Earth observation data is to store small scenes as individual files that are manageable on desktop computers or workstations, resulting in millions of gigabytes or petabytes of data being acquired. This approach makes the data difficult to work with on lengthy projects or large regional areas such as the Murray-Darling Basin, often resulting in significant and costly delays to research.
With new satellites expected to come online from countries like the United States over the next decade, the volume of data is expected to increase exponentially, producing multi-petabyte datasets.
Handling massive increases in data volume
Our researchers, in collaboration with Geoscience Australia and the National Computational Infrastructure (NCI), further developed the ‘Australian Geoscience Data Cube’ originally created by Geoscience Australia that houses and processes petabytes of satellite Earth observation data from multiple sensors.
Since 2017, several international partners (NASA, USGS, CEOS, UK’s Satellite Applications Catapult) have joined the partnership and the platform is now called the Open Data Cube. It provides an efficient and flexible programming interface that significantly simplifies access to freely available satellite data about our world.
Users are able to add their own algorithms into the analysis framework, and for many applications it allows parallel computation across thousands of processes and a petabyte of data, enabling scientists to use the Earth observation data more efficiently, measuring changes or mapping new aspects of the environment and surrounding oceans.
California Data Cube
There is clearly a need for this type of technology to address the challenges in areas such as wildfire recovery and mitigation, forest health monitoring, agriculture, mining, and climate change adaptation. We’re pleased to be partnering with The University of California, Davis, on the development of the Data Cube California Centre of Excellence delivering a number of key projects, along with NASA Langley and Amazon Web Services. To find out more, connect with the team.