APS News

December 2017 (Volume 26, Number 11)

Managing the Flood of Space Program Data

A web and marketing specialist has a solution for how to handle the copious data produced by space missions.

By Katherine Wright

In the sweltering sun of the New Mexico desert, Ben Feist held out his cell phone. Snap. The camera on a nearby scientific instrument captured a shot of the phone’s screen showing a clock. Click. Another instrument did the same.

The instruments’ operators, a handpicked team of NASA geologists and an active astronaut who lived in space for six months, were in the desert testing gadgets for future space explorers. The clock shots, Feist hoped, would provide a way to sync data in chronological order from different instruments. Feist came up with the idea while trying to piece together documents from the Apollo 17 moon-landing mission. But Feist doesn’t work for NASA; he’s a web-data specialist based in Canada and he didn’t expect to be here. "I never dreamed of being in a position where I could contribute to the challenge of helping humanity to leave Earth and visit another planet," said Feist.

Space exploration produces an abundance of data. In 1972, Apollo 17 astronauts Eugene Cernan and Harrison Schmitt snapped thousands of photographs, bagged 334 rock samples, and performed numerous experiments, including investigations of the moon’s tenuous atmosphere and the electrical properties of its rock. More recently, "robot geologist" Mars Rover has been making scientific observations with nearly a dozen instruments for the last 17 years. Knowing how these extensive datasets fit together is essential if they are to be properly interpreted.


Drone vehicle photo
Katherine Wright

Ben Feist and a colleague prepare a drone vehicle for studying data synchronization and management in a simulated space environment.


"These missions are extraordinarily complex," said Paul Niles, a planetary scientist at NASA’s Johnson Space Center who studies Mars’ geochemistry. "Being able to quickly and easily understand what happened in a mission, what data were collected [and where], and how that data fits into everything else, that’s key … but we don’t do it very well."

For human missions the turnaround time is particularly important. Astronauts should be able to quickly review the day’s data so they can plan for their next spacewalk, said Jacob Bleacher, a geoscientist at NASA’s Goddard Space Flight Center, who tests geological instruments for use in space by humans. Today they can’t. "We don't have a way for the science team and the crew to handle that data in real time," said Bleacher.

So how does NASA manage their data now? Missions like NASA’s Mars Science Laboratory dump datasets into a repository called The Planetary Data System (PDS), which according to Niles is "very hard to navigate." Unless you are on the team that took the data, finding and interpreting files on PDS is overly difficult. It can take months to figure out the where, when, and what for each file, said Niles, providing a significant barrier to data analysis. "It’s just a couple of steps above raw data," he said. Both Niles and Bleacher think that linking datasets with transcripts, audio recordings, photographs, and videos via timestamps could solve this problem, particularly in a tool that allows you to move seamlessly through documents chronologically or geographically. This is exactly what Feist created with Apollo17.org.

Apollo17.org connects over 300 hours of audio, 22 hours of video, and 4200 photographs from this 1972 moon landing, enabling viewers to experience the mission in real time from launch to splashdown. You can simultaneously watch, listen, and read along as the astronauts journey through space and explore the moon’s surface. Feist also added in images from the Lunar Reconnaissance Orbiter and data from samples the astronauts returned to Earth, which are accessible at the click of a mouse. "You can watch the moment Jack picked up the sample, and see images of the rocks, without having to go do a bunch of legwork to figure out when that might have happened and which video file you need to look at," said Feist. "Suddenly you have a research tool."

Feist’s work shows the effectiveness of timestamps to organize information. But for it to be a useful research tool the streams of data need to sync automatically, without hours of manual intervention—Apollo17.org was five years in the making, albeit mostly evenings and weekends. Then team members can instantly explore and analyze the footage and measurements being collected with little to no effort.

To field-test (near) real-time synchronization of data, Bleacher invited Feist to the desert. There he evaluated a data linkup protocol while Bleacher’s team carried out simulated moonwalks around a volcanic crater. The protocol involved capturing a shot of a highly accurate clock, such as those found on phone apps, in the viewfinder of each instrument. The shots embedded "timestamp" markers in each data stream that Feist could use to quickly arrange measurements and recordings in chronological order.

By the end of the week, Feist had stitched together video footage from the first field day (cameras were attached to the chests of the two members of astronaut "crew" carrying out the simulated space walks, and one was attached to a drone, capturing an aerial view of their journey). Since then, he has added in still images the crew shot. In his mock-up website, the recordings play simultaneously, side-by-side on the screen, while the path the team traversed overlays a map of the desert giving the crew’s spatial location. Bleacher adds that now the team can sit down and replay any part of the day they need to, a luxury they’ve never had in the past.

"Ben’s approach, his methodology is pretty outstanding," said Noah Petro, a planetary geologist also at NASA’s Goddard Space Flight Center, who was responsible for bringing Feist and Bleacher together. "It opens the door for interpretation of data in near real time when we are in the field, or if humans are doing something in space."

With their proof-of-principle data link-up running, Bleacher, Niles, and Feist hope to take this project further and have submitted a grant proposal for funding. Bleacher said this tool has a wide range of possible uses. It could find applications as an educational outreach mechanism for NASA in the form of Apollo17.org-like websites, as a data collection teaching aid in classrooms, or as a tool to coordinate the efforts of rescue crews and aid organizations in regions ravaged by natural disasters. Bleacher added that "The options for use of this type of software are immense."

Katherine Wright is an associate editor of Physical Review Letters and a contributing editor of Physics.

December 2017 (Volume 26, Number 11)

Table of Contents

APS News Archives

Contact APS News Editor


Articles in this Issue
What You Need to Know: APS and SCOAP3
APS Inventories Its Carbon Footprint
Charting a Future for U.S. Physics
YouTube’s Physics Girl
A Physicist Pushes for Interstellar Travel
Robert Henry Bragg, Jr. 1919-2017
Managing the Flood of Space Program Data
International News
This Month in Physics History
News from the APS Office of Public Affairs
The Back Page

©1995 - 2017, AMERICAN PHYSICAL SOCIETY
APS encourages the redistribution of the materials included in this newspaper provided that attribution to the source is noted and the materials are not truncated or changed.

Editor: David Voss
Contributing Correspondent: Alaina G. Levine
Publication Designer and Production: Nancy Bennett-Karasik