A diagnostic tool that enables astronauts to troubleshoot anomalies by encouraging best practices, critical thinking and documentation.
Role

Experience Design
Visual Design
Research

Company

NASA
(Capstone at Carnegie Mellon University)

Duration

8 months (2020)

Team

Aditi Magal, Katie McTigue, JT Aceron, Megan Parisi, Nathan Barnhart

PROBLEM

Astronauts rely heavily on Mission Control (a 100+ personnel ground team) on earth to solve anomalies.
However, a mission to Mars means long communication delays and blackouts, which means astronauts will have to work more autonomously from Mission Control while diagnosing problems.
How do we get the expertise of 100+ experts on earth to 4 astronauts in space?

RESEARCH

To understand how astronauts currently diagnose problems and the future of troubleshooting, we wanted to get a holistic picture of the problem space. We used 4 research methods:

Background

20 papers
4 professors
3 commercial space employees

Generative

22 interviews
3 contextual inquiries

Empathetic

4 playtests
105 diary study participants

Evaluative

8 storyboards
6 validated needs

We were incredibly lucky to go to the Johnson Space Center in Houston, the home of Mission Control! We sat in on a flight controller training session for some contextual inquiry, saw Mission Control, went inside a full scale Mockup of the International Space Station (ISS) and talked to subject matter experts (Flight Controllers, Astronaut Trainers, Flight Directors, MER Managers and CAPCOMs)

We interviewed and user tested our prototypes with 5 astronauts!!
We wanted to see how experts diagnose anomalies in other problem spaces, so we turned to Analogous domains by interviewing and surveying HVAC technicians, miners, electricians, auto mechanics, pilots and computer engineers.

Evaluating Needs

Through our research, a lot of astronaut needs started to emerge. We tested these needs by creating storyboards and presenting them to astronauts! Here are some examples of our storyboards:

How might we keep the crew going when things get tough? Need Validated

How might we condense information into a single screen? Need Validated

How might we make safety and engineering data more available for just-in-time training? Need Validated

How might we optimize diagnostic schematics? Need Validated

How might we make it easier for astronauts to follow multiple procedures? Need Validated

How might we manage conversations with long delays? Need not validated

How might we help people practice troubleshooting best practices even when they get frustrated and suffer from the “space stupids?” Need Validated

How might we help astronauts remember complex systems training? Need Validated

How might we update the concept of procedures for unanticipated anomalies? Need Validated

How might we make authority clear? Need not validated

How might we help astronauts characterize risk? Need not validated

This process worked really well, because astronauts were very honest. They told us when a need was valid, and when something wasn’t our strongest idea.

More details on my research
INSIGHTS

We arrived at insights by organizing information from interviews and contextual inquiries in multiple affinity maps. Based on our insights we created a model to enable successful troubleshooting in the context of future space missions.

ABCD Model


Access to resources
Astronauts need reference material to make the right call and reduce dependency on memory- Telemetry, procedures, schematics.

“Crew members only look at instantaneous data, so their situational awareness is much less.”
-Astronaut trainer

Best practices
Astronauts need to be nudged to follow best practices in high stress environments.

“Even the most experienced engineers break the rules sometimes.”
- Electrical Engineer

Critical thinking
Astronauts need to think like flight controllers, ask the right questions at the right time and think through downstream implications.

“Crew is trained to adhere to procedures, you don't want to operate outside of that."
-Astronaut on Soyuz-10

Documentation
Mission control won’t be replaced in the near future. Documentation is necessary to allow MCC to assist.

“In high stress situations, the crew has more situational awareness, and the ground does not provide good judgement.”
- Flight Controller

Based on our insights and a reframing exercise, we crafted a narrowed problem statement:

How might we situate diagnostic tools in a workflow to support astronauts in thinking clearly, rationally, and critically?

USER TESTING

To answer our how might statement, we ran 3 user testing sessions with prototypes of increasing fidelity.
We tested our prototypes with 3 astronauts and 7 subject matter experts. In each session, we walked the participant through the prototype, highlighting its various interactions and capabilities.

User testing revealed that astronauts wanted:

Automation
Any technology that can automate routine planning items and take the burden off the astronauts is welcome.

Flexibility
Astronauts want the ability to create non-linear yet connected plans.

Privacy
Astronauts are hesitant to send down half-baked thoughts to mission control and wanted the ability to correct an uninformed message.


SOLUTION

TALOS is a diagnostic tool that enables astronauts to troubleshoot anomalies by encouraging best practices, critical thinking and documentation.
TALOS allows astronauts to gather resources, create and execute a plan of action, and communicate progress with mission control. We envision astronauts using it on iPads as they gather and analyze information and on a laptop for in-depth planning and documentation.

TALOS has three main views:

Gather View
Displays resources like procedures, schematics, telemetry, and personal notes.

Plan View
A Kanban-style project management area, which uses flexible cards and columns to visualize workflow.

Split View
Simultaneously view the gather and plan views. Resources from the gather view can be dragged directly into cards, giving them additional context.

Cards

The front of the cards contain the card type, card title, time the card was created, and ID of the crew member who created the card.

“This is really what these cards do, they give you a bit of a mental model, both in functional space and time.”
- Astronaut on STS-85

Access to Resources Best Practices Critical Thinking Documentation


Back of card

Records more in-depth information about the observation or action. Astronauts are prompted to consider downstream implications of the card and potential times to effect.

“I think what you've done is created a space that's much more multi-sensory, that makes the picture that's being painted much more intuitive and complete than typing it out on a form.”
- NEEMO Lead

Critical Thinking


Time to effect card

TALOS pre-populates the unknown column of each swimlane with a card prompting crew members to consider time to effect (the time until a particular effect is expected to manifest) and near-future impacts.

“It’s like playing chess. Thinking two moves ahead is one thing. Thinking four moves ahead is a much more complicated thing.”
- Astronaut on STS-85

Critical Thinking


Columns

The planning board has 5 pre-populated columns: unknowns, knowns, potential actions, completed actions, and results. Keeping track of every action and its result, allows crew to test one thing at a time.

“The ability to communicate what is known and unknown, and how to make the unknown known, is critical.”
- Flight Controller

Best Practices

Swimlanes

Swimlanes function similarly to branches on a fault tree. Astronauts are encouraged to follow a path and lop it off if it is unrelated, this makes the path more clear.

“Basically, you’re helping me do a storyboard, lopping off branches on the fault tree.”
- Astronaut on Expedition 22

Best Practices

Capturing Media

Using the bar at the bottom of the board, crew members can quickly capture audio, video, and photos on their iPad and add media to the board.

“Yeah, it seems to me that it makes the kind of stuff you’d want to do anyway easier to manipulate and move around, and that’s all really good.”
- Astronaut on STS-127

Documentation


Timers

Crew members can set named timers which they can drag into specific cards. This associates them with a certain task.

“We have our timers, the little white egg timers... a timer goes off and we’re all looking at each other. What was that for?”
- Astronaut on STS-127

Critical Thinking


Voice to text recording

During user testing, crew members mentioned time and time again how difficult and time- consuming typing can be in zero-gravity circumstances. Document written records through voice-to-text technology.

“That can be very diagnostic to a ground team. So video and audio as well as the voice to text I think would be extremely useful and time-saving useful for everybody.”
- Astronaut on STS-127

Documentation


Easy communication with MCC

Send messages associated with a certain point in time using the “Send” button on the planning board. Receive messages from MCC in the form of cards.

“The thing I like the most is exactly that - the ability to click a button and now the ground is getting information!”
- Astronaut on STS-127

Documentation


VISUAL DESIGN

I was responsible for the interaction design (creating the entire prototype) and visual design of our UI interface.The colors were used purposefully to communicate how the plan and gather board function. Darker backgrounds for the gather board and lighter backgrounds for the plan board communicate difference between the two boards.
I created a design system on figma for our team to follow with reusable components.

Colors and typography

Buttons and Logo

The colors and fonts used also pass accessibility.


KEY TAKEAWAYS

Being resourceful (interviews with astronauts):
The client had told us that no past team had managed to talk to an astronaut. However, I was very determined to talk to my end-user, so I reached out to multiple astronauts on LinkedIn and email. Persistence and luck led me to find 5 astronauts who participated in our project!

Dealing with differing opinions (astronauts vs MCC):
Astronauts and MCC often had differing opinions- Level of autonomy, features required, etc. Since our end user is the astronaut, I placed more weight on their opinion while keeping MCC's view in mind.

Responsible for Interaction and Visual design:
My key contribution was designing functionality, building the entire prototype (interaction design) and visual design for the project. I built a design system from scratch and created reusable components.

The ABCD model is being used by the Human Systems Integration Division at NASA Ames Research Center!

Link to our team blog:

NASA X MHCI