Wave: Assisted Hearing

#Inclusive Design
#Design System

What's it about

CoDesigning

Inclusivity

Visual Identity

Concept Project

INST 704 - Inclusive Design

Dr. Hernisa Kacorri ↗︎

Team Members -
Esther Cho ↗︎ and Lining Wang ↗︎

Problem Statement

Our participant who is hard-of-hearing uses a hearing aid on a daily basis. The hearing aid sometimes cancels out foreground noise making it difficult for the participant to distinguish different sounds. This situation causes frustration and could even put the participant in danger.

Our Participant

We used a participatory design (Co-design) methodology to identify problem areas in a method where we collaborate with the participant who is hard of hearing.

We began with an interview with a participant who is hard of hearing to understand their lived experiences, identify pain-points, and articulate needs to develop a key problem statement. We also tried to understand the participant’s limitations, if any, and hear out any possible solutions or ideas they might have.

User interviews insights

The interview provided a good insight into the life of a hard-of-hearing person. We talked about their hobbies and everyday activities, and where the pain points come into play. We also talked about all the challenges they face on a daily basis, like the need for visual aids in crowded places, so they need to pay more attention.

User interviews insights

→ Settings like school and public transportation inconsistently have visual aids, such as captions and visual displays of information.
→ They need to pay closer attention to surroundings in public spaces.
→ In the past, the participant experienced their hearing aids cancelling out foreground sound when "automatic" sound filtering is on.
→ Due to their narrower range of hearing, they occasionally need to ask people to repeat themselves.
They have a harder time reading lips during a pandemic when everyone is wearing masks.
GOAL
With our findings, we generated the goal to improve noise detection and better source selection in crowded spaces.

Storyboarding

Based on our interview and insights, we decided to come up with a plan for a participatory design session to align our goals with our participants' wishes. We put down our objectives and agenda, and set up a problem statement on the basis of that: Participant has trouble with being able to tag different sounds and filter in/out people’s voices in crowded spaces. To break down the problem, we divided it into 3 subtasks: Personalized sound amplification and suppression, filtering out background noise, and to distinguish between relevant and irrelevant sound sources.
Provide an option to quickly set audio controls and good noise cancellations.
Provide an option to amplify and mute sound sources based on user preferences.

Participant CoDesign Session

We conducted a Co-design session with our participant to explore ideas that were in line with our goal. One idea that our participant came upon and liked was a slim circular wristband with access to sounds in a personal bubble around them. They also wanted to tag people they're talking to and be able to hone it to particular sounds.
Another idea we iterated on was access to a user's realized space using an antenna in an accessory form like necklace or earrings. The antennas would cover the space, and a wrist based device could control the audio. We brainstormed on different types of wrist/handheld devices, and forms for reconstructing the space digitally, 2D/3D.

Insights

From the Co-design session we determined and formulated our user’s needs properly, and changed our perspective in that direction.

Our participant wanted an interface to get sounds from a 3D space around the user, and allow access to front and back separately via separate controls, and especially being able to use dimension control in crowded places. Our main question from this was how to control the 3D space, especially the bounds and reach, and provide an interface to do so. We brainstormed and determined that a 2D interface would be much better for clearer audio space reconstruction. This would be based on a wristband based prototype for easier usage.

We also decided on not using the front and back audio control, since it could become counterproductive in crowded environments.

We did decide to add in tagging, allowing the user to identify and mark unknown sources and being able to focus in on a particular source. We experimented with the wristband features a little bit, working on interactions like controlling audio ranges and tagging. We also thought about dials for the band to control the audio, and a ball based interface for mapping the user’s realized space, and we also found a couple of problems with our wristband idea, like accidental triggering.

We decided to go with a hardware smartwatch prototype and a custom OS built around the features we thought of, with our participant.

Iterations

Wave v1

Iterated with our participant and professor.

􀇿 Scalability issues

Too many colors: one for each source

􀇿 Poor Consistency

Determine a better color and text consistency.


Wave v2

Iterated with our participant and professor.

􀇿 Bloated Interface.

A lot of things were too close to each other to actually be interfacable.

􀇿 Poor implementation of HIG

We went back to look at how apps are designed on the watch


Wave v3 switched to a landscape version

Iterated with our participant and professor.

􀇿 Reduce Textual Items

We were told to switch out textual items for icons to improving spacing and understandibility.

􀇿High Contrast Colors

Our colors had contrast issues and would be difficult to read for people with visual problems.

􀇿Decrease Text Type/More Consistency

We needed to build up a component library to decrease variation and improve consistency over the application.


Wave v4

Component library

􁁛  Less fonts, More icons.

Used SF Symbols for a consistent icon set.

􁁛  Colors only used for identification of sources.

Supported by icons for colorblind users. Everything else uses a duo-color theme and red for destructive actions.

􁁛  High Contrast Colors

All colors used conform to AA/AAA Large contrast.

􁁛  3 Font sizes

Header, Button and Body. Used a combination of weights, sizes and color to show heirarchy.

􁁛  Follows a design system

Built around a design system to ensure consistency and fluidity.


Wave v4

Final Screens

Wave v4

Closeups

Category Icons
Identifier icons get smaller and transparent in size by their volume and distance from the center on the map
Slider based Volume control for better usability than button based control.
List based view as an alternate option for map view, works better in crowded places
Expanding Volume control view in the same window.
Icons for quick identification.

Future Work

In future iterations of the design, we would research and/or co-design visualization techniques for and by blind people. We could also consider limiting the scope of the functionality on the watch by transferring what needs fine-grained motor control to a bigger screen, such as a smartphone or tablet, and/or developing equivalent applications for those platforms. Finally, we hope to add a captioning option for each sound source so that sound information can be consumed in multiple ways, which may be helpful for both hard of hearing and non-hard hearing users.

Key Takeaways

A longer co-design session would have been helpful as it felt like we were getting into the meat of an interesting design by the end of the session. If we had more time and resources for the design process, we would increase the number of co-design and feedback sessions, and have them build on each other in an iterative fashion.

Designing for watches / wearables! I learnt a lot by studying HIG guidelines for the Apple Watch, and how to utilise limited screen real estate for maximum effectiveness.

Outcome

The participant expressed that they overall liked the prototype and thought it would be useful in environments with heavy levels of sound. They also said that it was what they had in mind during the co-design session, which was helpful to hear since we based our design largely off of the insights from the session.

If you liked this project (or didn't), reach out to me↗︎ I'd love to talk to you!
Made with ❤️  by Aritro. 2022.