Managing the Flood of Space Program Data

Katherine Wright

Ben Feist and a colleague prepare a drone vehicle for studying data synchronization and management in a simulated space environment.

---

"These missions are extraordinarily complex," said Paul Niles, a planetary scientist at NASA’s Johnson Space Center who studies Mars’ geochemistry. "Being able to quickly and easily understand what happened in a mission, what data were collected [and where], and how that data fits into everything else, that’s key … but we don’t do it very well."

For human missions the turnaround time is particularly important. Astronauts should be able to quickly review the day’s data so they can plan for their next spacewalk, said Jacob Bleacher, a geoscientist at NASA’s Goddard Space Flight Center, who tests geological instruments for use in space by humans. Today they can’t. "We don't have a way for the science team and the crew to handle that data in real time," said Bleacher.

So how does NASA manage their data now? Missions like NASA’s Mars Science Laboratory dump datasets into a repository called The Planetary Data System (PDS), which according to Niles is "very hard to navigate." Unless you are on the team that took the data, finding and interpreting files on PDS is overly difficult. It can take months to figure out the where, when, and what for each file, said Niles, providing a significant barrier to data analysis. "It’s just a couple of steps above raw data," he said. Both Niles and Bleacher think that linking datasets with transcripts, audio recordings, photographs, and videos via timestamps could solve this problem, particularly in a tool that allows you to move seamlessly through documents chronologically or geographically. This is exactly what Feist created with Apollo17.org.

Apollo17.org connects over 300 hours of audio, 22 hours of video, and 4200 photographs from this 1972 moon landing, enabling viewers to experience the mission in real time from launch to splashdown. You can simultaneously watch, listen, and read along as the astronauts journey through space and explore the moon’s surface. Feist also added in images from the Lunar Reconnaissance Orbiter and data from samples the astronauts returned to Earth, which are accessible at the click of a mouse. "You can watch the moment Jack picked up the sample, and see images of the rocks, without having to go do a bunch of legwork to figure out when that might have happened and which video file you need to look at," said Feist. "Suddenly you have a research tool."

Feist’s work shows the effectiveness of timestamps to organize information. But for it to be a useful research tool the streams of data need to sync automatically, without hours of manual intervention—Apollo17.org was five years in the making, albeit mostly evenings and weekends. Then team members can instantly explore and analyze the footage and measurements being collected with little to no effort.

To field-test (near) real-time synchronization of data, Bleacher invited Feist to the desert. There he evaluated a data linkup protocol while Bleacher’s team carried out simulated moonwalks around a volcanic crater. The protocol involved capturing a shot of a highly accurate clock, such as those found on phone apps, in the viewfinder of each instrument. The shots embedded "timestamp" markers in each data stream that Feist could use to quickly arrange measurements and recordings in chronological order.

By the end of the week, Feist had stitched together video footage from the first field day (cameras were attached to the chests of the two members of astronaut "crew" carrying out the simulated space walks, and one was attached to a drone, capturing an aerial view of their journey). Since then, he has added in still images the crew shot. In his mock-up website, the recordings play simultaneously, side-by-side on the screen, while the path the team traversed overlays a map of the desert giving the crew’s spatial location. Bleacher adds that now the team can sit down and replay any part of the day they need to, a luxury they’ve never had in the past.

"Ben’s approach, his methodology is pretty outstanding," said Noah Petro, a planetary geologist also at NASA’s Goddard Space Flight Center, who was responsible for bringing Feist and Bleacher together. "It opens the door for interpretation of data in near real time when we are in the field, or if humans are doing something in space."

With their proof-of-principle data link-up running, Bleacher, Niles, and Feist hope to take this project further and have submitted a grant proposal for funding. Bleacher said this tool has a wide range of possible uses. It could find applications as an educational outreach mechanism for NASA in the form of Apollo17.org-like websites, as a data collection teaching aid in classrooms, or as a tool to coordinate the efforts of rescue crews and aid organizations in regions ravaged by natural disasters. Bleacher added that "The options for use of this type of software are immense."

Katherine Wright is an associate editor of Physical Review Letters and a contributing editor of Physics.