Research Showcases

Designing human-AI and multimodal interactions to make presentation slideshow reading and editing accessible.

Zhuohao (Jerry) Zhang,
Gene S-H Kim,
Jacob O. Wobbrock

To enable apps to respond to users’ situated abilities, we created the Ability-Based Design Mobile Toolkit (ABD-MT). ABD-MT integrates with an app’s user input and sensors to observe a user’s touches, gestures, physical activities, and attention at runtime, to measure and model these abilities, and to adapt interfaces accordingly. Conceptually, ABD-MT enables developers to engage with a user’s “ability profile,” which is built up over time and inspectable through our API.

Junhan Kong,
Mingyuan Zhong,
James Fogarty,
Jacob O. Wobbrock

Ability-Based Design Toolkit poster

With the goal of improving users’ experiences with assistive technologies in public libraries, our research found that JAWS is a widely used technology for users with disabilities. However, insufficient library staffing levels hinder individualized assistance, crucial for managing JAWS’s steep learning curve, complex interface, and extensive key memorization. Our project aims to develop a game-like solution akin to typing games, fostering muscle memory for efficient navigation. Drawing from user feedback and insights, we will select essential commands for specific tasks, like “Reading Text,” to simplify the learning process.

Fatima Ateeque,
Fernando Jimenez,
Julie Ganbold,
Vishrut Agarwal

Currently, hand function assessment (e.g., joint range of motion) in a clinical setting is done with low-resolution tools and oftentimes in a subjective manner that is time-consuming. With a camera-based interface, we wanted to improve the speed of collecting information about patient’s hand function, improve repeatability and objectivity, and enhance result presentation for both patients and clinicians.

Congyi Zhu,
Alexandra Portnova,
Adria Robert-Gonzalez,
Emily Boeschoten

Given the increasing interest of research on deceptive design patterns, it is important to examine how deceptive design impacts people who use visual accessibility technology (e.g., screen readers, screen magnification, braille displays). We present an interview and subsequent diary study with 16 people who use visual accessibility technology to better understand their experiences encountering and navigating deceptive design patterns in online services. We report participant experiences with six deceptive design patterns, direct and indirect exacerbated consequences of those deceptive designs, and impacts of deceptive design from a perspective of consequence-based accessibility. We finally propose that accessibility tools could help address deceptive design patterns by offering higher-level feedback to well-intentioned designers.

Aaleyah Lewis,
Jesse J. Martinez,
Maitraye Das,
James Fogarty,

We are designing a sensory playroom for children with chronic venous insufficiency (CVI) and cerebral palsy. The current options on the market are not portable and costly. We aim to create a space for these children to play and interact independently with, where they can also learn and develop their motor skills.

LIa Mar-Lundeen,
Makena Lyle,
Becca Caulton,
Catherine Ton,
Katelyn Doan

We have designed a modular flow-limiting cup for use by individuals who have difficulty swallowing liquid.

Motivated by the challenge faced by children with conditions such as Down Syndrome or dysphagia, who struggle with traditional drinking methods. Aimed to address the limitations of existing products which are often nondurable, complex, and stigmatizing. Driven by a deep understanding that independence in daily activities such as drinking can significantly boost self-esteem and social acceptance among users in all demographics.

Spencer Madrid,
Hari Rathnam,
Myky Ho,
Oliver Sommer,
Chaelynn Lim,
Sunny Dighe,

Flow-limiting cup poster

Videoconferencing usage has surged in recent years, but current platforms present significant accessibility barriers for millions of d/Deaf or hard of hearing people worldwide. Informed by prior work examining accessibility barriers in current videoconferencing platforms, we designed and developed a videoconferencing platform to facilitate communication in mixed hearing groups.

Anant Mittal,
Meghna Gupta,
Roshni Poddar,
Tarini Naik,
SeethaLakshmi Kuppuraj,
James Fogarty,
Pratyush Kumar,
Mohit Jain,

Paraplegic and quadriplegic users use mouth mounted styluses to interface with tablets and other touch screen devices. Our holder makes it so users have the ability to mount and unmount their styluses without assistance.

Mark Fernandez,
Sawani Deshmukh,
Emma Martin,
Laura Oliveira,
Devanshi Desai,
Sam Fleet

Mouth Stylus Holder poster

Video games often pose accessibility barriers to gamers with disabilities, yet there is no standard method for identifying which games have barriers, what those barriers are, and whether and how they can be overcome. In this project, we present the results of an interview study with gamers with disabilities to explore how game developers and distributors can support the process of finding and getting set up with accessible games. We also explore how experiences with access barriers inform new styles of gameplay, through the concepts of “access difficulty” and “disabled gaming.”

Jesse J. Martinez,
Jon E. Froehlich,
James Fogarty

We are exploring the use of a smartphone touchscreen as a VR controller for BLV users. Our approach leverages the phone’s familiar interface, multi-touch gestures and vibrotactile feedback to improve accessibility and immersion in VR environments. We focus on improving scene understanding through finger-driven, touchscreen exploration, which we have termed, scene reading, and enabling additional object interaction and locomotion.

Melanie Kneitmix,
Jacob O. Wobbrock

Our team aims to provide visually impaired children affordable books and a reusable spine through the use of sensory enhanced graphics while focusing upon play, behavioral models, and positive childhood development within storytelling.

Tiffany Matsushima,
Nupur Gorkar,
Amogh Bantwal,
Rohan Aggarwal,
Aadya Goyal,
Emily White,
Kate Glazko

This research involved the creation and evaluation of a therapeutic toy named the “Switch Kit,” designed for young children with medical complexities. The kit allows family members and clinicians to customize switches tailored to the unique needs of each child.

Mia Hoffman,
Kate Bokowy,
Alisha Bose,
Tiffany Li,
Heather A. Feldner,
Katherine M. Steele

Switch Design Poster

People with Parkinson’s disease are prone to experiencing cognitive decline. External tools, like electronic aids, are a promising way to support cognition. This study examined the usability and feasibility of a cognitive aid, IndiAide^©, in people with Parkinson’s disease with a range of cognitive functioning.

Dale Summers,
Kristie Spencer

Parkinson’s Cognitive Aid Poster

Parkinson’s Cognitive Aid project summary

Our team has designed and built an automated wheelchair umbrella device for the Permobil Corpus M3 powered wheelchair. At the push of the adapted button a canopy will extend over the wheelchair, covering both the user and any water sensitive controls. Our umbrella design prioritizes functionality and user comfort making it a reliable tool for wheelchair users in various weather conditions.

Courtney Cho,
Annie Yao,
Ella Jung,
Jimmy Le,
Aidan Frondozo

Wheelchair umbrella poster

There has been little focus on the intersection of race and disability within accessibility research, missing out on the full nuance of marginalized and “otherized” groups. In this upcoming ASSETS paper, we offer a framework that outlines opportunities for research to engage the intersection of race and disability. We present a series of case studies that exemplifies engagement at this intersection throughout the course of research, along with reflections on teaching at this intersection. This work highlights the value of considering how constructs of race and disability work alongside each other within accessibility research studies, designs of socio-technical systems, and education.

Christina N. Harrington,
Aashaka Desai,
Aaleyah Lewis,
Sanika Moharana,
Anne Spencer Ross,
Jennifer Mankoff

Race, Disability & A11y poster

Race, Disability & A11y project summary