Harnessing VOICE-ENABLED DIGITAL ASSISTANTS to empower patients and HOSPITAL STAFF

What began as a thought-experiment to explore how Voice User Interfaces (VUIs) can be incorporated into the healthcare ecosystem, evolved into a collective passion-project that delved into the current state of healthcare, the social and emotional connections that voice interactions can have on a patient’s healing, and how technology can have a meaningful and positive impact on hospital patients and those that care for them.

• Accelerated the agency's practical knowledge in VUIs and related tech

• Invited to present and demo model at Seattle Interactive Conference

• Brought Virginia Mason Foundation to the table as a client and partner

The challenge

Each year, millions of people find themselves in a hospital room.

During that time, they may be confined to bed, have limited mobility, or experience movement-related pain. Moreover, a hospital stay can feel socially isolating and anxiety inducing, as the patient has little control over his/her environment and situation. 

A study published in the Journal of Health Environments Research and Design found that the level of control patients feel they have can affect how quickly they heal — and that has ripple effects beyond the patient. It affects the strain on loved ones, the cost of care for providers and insurers, the amount of time and energy doctors and nurses can allocate to individual patients, and more.

Which led to the question: how can technology be used to drastically improve the experience for patients, and staff, in a hospital setting?

When we’re sick and feeling vulnerable, it’s especially important to feel in control of our surroundings… Each change [to the hospital experience] can improve the patient and family experience by reducing unnecessary stress and anxiety and enhancing the healing process.

Dr. Emily Patterson, Ph.D. Ohio State University Wexner Medical Center

Design for one, solve for all

Taking a cue from Microsoft's Inclusive Design manifesto, my "lab partners" Lulu, Antonio, and I focused first on the hospital patient — specifically those with moderate to severe mobility limitations — to align on a targeted and reasonably scoped value proposition.

The healthcare experience revolves around patients and their needs, but it extends beyond the hospital ward and involves many people in addition to the patient.  Introducing voice-based and AI-powered devices to patients meant we can also introduce them to hospital staff, loved ones, care advocates, and insurance providers.

Patient vitals could be shared securely with hospital staff, communication between patients and those charged with their care could be streamlined, and simple tasks could be augmented or automated to save time, energy and money.

I spearheaded a journey to experiment with the untapped potential of voice-based UI, envisioning its role in enhancing the in-patient care experience, reducing healthcare costs, and fostering improved communication.

As my lab partner and I delved deeper into the research, we unearthed pivotal questions:

• In what ways can AI help improve the healthcare system?
• How do we effectively address human’s inherent suspicion regarding “machines that listen, speak and think”?
• How can AI help doctors, nurses and caregivers perform their jobs at higher effectiveness?
• How effective can AI be in helping reduce cognitive load, reduce staff stress and fatigue, reduce accidents, and increase overall patient wellness?
• Other than short-term care, like hospitals, could an AI be introduced to long-term care, such as nursing homes or assisted living spaces, and how could AI help people stay in their own homes longer?

I also brainstormed with the broader org how best to present the proof of concept — the interactive nature of the story we were crafting demanded something more than "just another pretty slideshow"...

...Then I JOKINGLY SAID “WE SHOULD BUILD diorama OF A HOSPITAL ROOM” and everyone LOST THEIR F#ING MINDS.

It's only a model...

Starting with what was readily available around the studio and what folks had brought in from home, we divided into three strike-teams — Story, Tech, and Build — and were able to rough out a scale-model hospital room in diorama form in which we could ask the voice-agent (Alexa in this case) to raise or lower the angle of the bed, turn the room lights on or off, or page a nurse.

As we worked, we kept our focus on the core idea that uniting inclusive design with smart devices can significantly improve patient care.

Our diorama prioritizes patient control, quality of care, and connection to loved ones, while also addressing the needs and challenges of the hardworking hospital nursing staff.

We believe that the marriage of inclusive design and connected smart devices has the power to significantly improve the quality and efficiency of patient care.

Lulu (Xiao) Hoogerheyde, Strategist

The evolutionary pace of AI-enabled devices and experiences is entirely dependent upon the pace of acceptance by humans. Voice-based UI is the next logical step in the evolution of fully-formed artificial intelligences, but if trust and ethics are not baked in from the start, humans will quickly and summarily reject this technology.

Me

The rest of the story...

My plan was to continue and expand on this work — and include the fine folks from Virginia Mason Foundation who'd attended our session at Seattle Interactive Conference 2017 and found our work fascinating (and worthy of investment).

But in 2018 two seismic shifts happened.  In January of that year, I was hired as Head of UX for Zonar, so I handed this passion-project off to my lab partners.  Shortly thereafter, Smashing Ideas was acquired by Luxoft and the work was shelved.

But I'm proud of the work my team and I did to show that the ideal hospital room experience empowers patients to transition easily from a state of dependence to independence as they prepare to continue their healing at home.

Through our work on the diorama, we were able to:

• Validate our premise and identify ways that voice-based UI and AI-powered devices can reduce staff stress and fatigue
• Increase operational effectiveness
• Improve patient safety
• Reduce patient anxiety
• Improve overall quality of care
• Reduce cognitive load
• Increase overall service satisfaction for hospital staff, insurers, patients and caregivers

⬆ Slides from the SIC2017 session (click to view)

Promotional videos for agency case study  ⬇