Square Kilometre Array (SKA) South Africa has announced plans to work with multinational tech giant IBM to develop a next-generation “big data” platform for analyzing massive volumes of radio astronomy data, allowing astronomers to observe objects in space more effectively.
In a statement on Wednesday, SKA South Africa said the proposed joint research project would combine available radio astronomy analysis software with machine-learning techniques currently under development at IBM Research.
“Processing big data volumes, whether from scientific instruments, environmental sensors, or international communication and commerce, will require extreme automation and self-learning capability by the software of the future,” SKA South Africa said.
“Beginning with components from IBM’s Infosphere software for big data and IBM’s SPSS software for predictive analysis, the initial phase is intended to programme computers to self-learn, adapt, and fine-tune the analysis of radio telescope data under the watchful eye of an astronomer.”
Africa vies for Square Kilometre Array
South Africa, allied with eight other African countries, is competing against Australia (allied with New Zealand) to host the €1.5-billion Square Kilometre Array, an instrument 50-100 times more sensitive and 10 000 times faster than any radio imaging telescope yet built.
The international science funding agencies and governments involved in the international SKA consortium are due to make an announcement – possibly on the final winning bid – on 4 April, with construction likely to start in 2016 and take place in phases over several years, with completion by about 2022.
South Africa is currently building a 64-dish precursor instrument for the SKA, the Karoo Array Telescope (also known as the MeerKAT) which, regardless of whether South Africa wins the SKA bid, will be a powerful scientific instrument in its own right – as will Australia’s SKA precursor, the 36-dish Pathfinder, which is currently under construction.
Massive data analysis – the challenge for SKA
The analysis of MeerKAT and Pathfinder data will be a major challenge – and solving this challenge would overcome a major hurdle for the SKA, whose data rates will be huge, rivaling the world’s internet traffic.
The MeerKAT, the Pathfinder, and following them the SKA, will generate massive volumes of data that will need to be combined to make detailed images of radio emission from distant objects like black holes, spinning neutron stars, planets, galaxies – even primeval gases that existed before the galaxies were formed, as observed at the edge of the visible universe.
“A number of subtle effects need to be corrected to make clear and accurate images from interferometers like MeerKAT,” says SKA South Africa’s Dr Jasper Horrell. “These include variations in the instrument itself [as well as] effects such as those introduced by the earth’s ionosphere.
‘More intelligent software is needed’
“More intelligent software is needed to enable astronomers to process and analyze the enormous data rates that will be produced by MeerKAT and future radio telescopes,” says Horrell.
The current method of analysis requires direct interaction with a computer for hours or days before the images can be used for research purposes. This practice is not only time-consuming, but it also requires experienced astronomers, making the radio sky accessible to only a few experts.
The proposed project with IBM would aim to “to teach a computer to make perfect images on its own,” says Dr Alain Biem, an IBM Researcher who specializes in exploratory stream analytics. “A software platform like this may assist in enabling large survey instruments like MeerKAT to process the trillions of bits of data per second they receive and make it available to astronomers around the world.”
