UW News: How an assistive-feeding robot went from picking up fruit salads to whole meals

November, 2023

In tests with this set of actions, the robot picked up the foods more than 80% of the time, which is the user-specified benchmark for in-home use. The small set of actions allows the system to learn to pick up new foods during one meal. UW News talked with co-lead authors Gordon and Nanavati, both doctoral students in the Paul G. Allen School of Computer Science & Engineering, and with co-author Taylor Kessler Faulkner, a UW postdoctoral scholar in the Allen School, about the successes and challenges of robot-assisted feeding. The team presented its findings Nov. 7 at the 2023 Conference on Robotic Learning in Atlanta.

An assistive-feeding robotic arm attached to a wheelchair uses a fork to stab a piece of fruit on a plate among other fruits.

The Personal Robotics Lab has been working on robot-assisted feeding for several years. What is the advance of this paper?

Ethan K. Gordon: I joined the Personal Robotics Lab at the end of 2018 when Siddhartha Srinivasa, a professor in the Allen School and senior author of our new study, and his team had created the first iteration of its robot system for assistive applications. The system was mounted on a wheelchair and could pick up a variety of fruits and vegetables on a plate. It was designed to identify how a person was sitting and take the food straight to their mouth. Since then, there have been quite a few iterations, mostly involving identifying a wide variety of food items on the plate. Now, the user with their assistive device can click on an image in the app, a grape for example, and the system can identify and pick that up.

Taylor Kessler Faulkner: Also, we’ve expanded the interface. Whatever accessibility systems people use to interact with their phones — mostly voice or mouth control navigation — they can use to control the app.

EKG: In this paper we just presented, we’ve gotten to the point where we can pick up nearly everything a fork can handle. So we can’t pick up soup, for example. But the robot can handle everything from mashed potatoes or noodles to a fruit salad to an actual vegetable salad, as well as pre-cut pizza or a sandwich or pieces of meat.

In previous work with the fruit salad, we looked at which trajectory the robot should take if it’s given an image of the food, but the set of trajectories we gave it was pretty limited. We were just changing the pitch of the fork. If you want to pick up a grape, for example, the fork’s tines need to go straight down, but for a banana they need to be at an angle, otherwise it will slide off. Then we worked on how much force we needed to apply for different foods.

In this new paper, we looked at how people pick up food, and used that data to generate a set of trajectories. We found a small number of motions that people actually use to eat and settled on 11 trajectories. So rather than just the simple up-down or coming in at an angle, it’s using scooping motions, or it’s wiggling inside of the food item to increase the strength of the contact. This small number still had the coverage to pick up a much greater array of foods.

We think the system is now at a point where it can be deployed for testing on people outside the research group. We can invite a user to the UW, and put the robot either on a wheelchair, if they have the mounting apparatus ready, or a tripod next to their wheelchair, and run through an entire meal.

For you as researchers, what are the vital challenges ahead to make this something people could use in their homes every day?

EKG: We’ve so far been talking about the problem of picking up the food, and there are more improvements that can be made here. Then there’s the whole other problem of getting the food to a person’s mouth, as well as how the person interfaces with the robot, and how much control the person has over this at least partially autonomous system.

TKF: Over the next couple of years, we’re hoping to personalize the robot to different people. Everyone eats a little bit differently. Amal did some really cool work on social dining that highlighted how people’s preferences are based on many factors, such as their social and physical situations. So we’re asking: How can we get input from the people who are eating? And how can the robot use that input to better adapt to the way each person wants to eat?

Amal Nanavati: There are several different dimensions that we might want to personalize. One is the user’s needs: How far the user can move their neck impacts how close the fork has to get to them. Some people have differential strength on different sides of their mouth, so the robot might need to feed them from a particular side of their mouth. There’s also an aspect of the physical environment. Users already have a bunch of assistive technologies, often mounted around their face if that’s the main part of their body that’s mobile. These technologies might be used to control their wheelchair, to interact with their phone, etc. Of course, we don’t want the robot interfering with any of those assistive technologies as it approaches their mouth.

There are also social considerations. For example, if I’m having a conversation with someone or at home watching TV, I don’t want the robot arm to come right in front of my face. Finally, there are personal preferences. For example, among users who can turn their head a little bit, some prefer to have the robot come from the front so they can keep an eye on the robot as it’s coming in. Others feel like that’s scary or distracting and prefer to have the bite come at them from the side.

A key research direction is understanding how we can create intuitive and transparent ways for the user to customize the robot to their own needs. We’re considering trade-offs between customization methods where the user is doing the customization, versus more robot-centered forms where, for example, the robot tries something and says, “Did you like it? Yes or no.” The goal is to understand how users feel about these different customization methods and which ones result in more customized trajectories.

What should the public understand about robot-assisted feeding, both in general and specifically the work your lab is doing?

EKG: It’s important to look not just at the technical challenges, but at the emotional scale of the problem. It’s not a small number of people who need help eating. There are various figures out there, but it’s over a million people in the U.S. Eating has to happen every single day. And to require someone else every single time you need to do that intimate and very necessary act can make people feel like a burden or self-conscious. So the whole community working towards assistive devices is really trying to help foster a sense of independence for people who have these kinds of physical mobility limitations.

AN: Even these seven-digit numbers don’t capture everyone. There are permanent disabilities, such as a spinal cord injury, but there are also temporary disabilities such as breaking your arm. All of us might face disability at some time as we age and we want to make sure that we have the tools necessary to ensure that we can all live dignified lives and independent lives. Also, unfortunately, even though technologies like this greatly improve people’s quality of life, it’s incredibly difficult to get them covered by U.S. insurance companies. I think more people knowing about the potential quality of life improvement will hopefully open up greater access.

Additional co-authors on the paper were Ramya Challa, who completed this research as an undergraduate student in the Allen School and is now at Oregon State University, and Bernie Zhu, a UW doctoral student in the Allen School. This research was partially funded by the National Science Foundation, the Office of Naval Research and Amazon.

For more information, contact Gordon at ekgordon@cs.uw.edu, Nanavati at amaln@cs.uw.edu and Faulkner at taylorkf@cs.washington.edu.


Excerpted and adapted from the UW News story by Stefan Milne.

Off to the Park: A Geospatial Investigation of Adapted Ride-on Car Usage

November 7, 2023

Adapted ride-on cars (ROC) are an affordable, power mobility training tool for young children with disabilities. But weather and adequate drive space create barriers to families’ adoption of their ROC. 

CREATE Ph.D. student Mia E. Hoffman is the lead author on a paper that investigates the relationship between the built environment and ROC usage.

Mia Hoffman smiling into the sun. She has long, blonde hair. Behind her is part of the UW campus with trees and brick buildings.

With her co-advisors Kat Steele and Heather A. Feldner, Jon E. Froehlich (all three CREATE associate directors), and Kyle N. Winfree as co-authors, Hoffman found that play sessions took place more often within the participants’ homes. But when the ROC was used outside, children engaged in longer play sessions, actively drove for a larger portion of the session, and covered greater distances.

Accessibility scores for the sidewalks near a participant’s home on the left and the drive path of the participant on the right. Participant generally avoided streets that were not accessible.

Most notably, they found that children drove more in pedestrian-friendly neighborhoods and when in proximity to accessible paths, demonstrating that providing an accessible place for a child to move, play, and explore is critical in helping a child and family adopt the mobility device into their daily life.

Community Partner Spotlight: PAVE

November 8, 2023

CREATE is pleased to work with PAVE (Partnerships for Action | Voices for Empowerment) to help guide our efforts and shape solutions around the needs and limitations of accessible technology. They’ve supported our grant applications, shared opportunities for participation in CREATE research projects with their community, and published CREATE research on the importance of self-initiated mobility for children, particularly children with disabilities. 


PAVE logo, with the V in a light green color and stylized to look like a flower.

PAVE’s mission is to provide support, training, information, and resources to empower and give voice to individuals, youth, and families living with disabilities throughout Washington State.


“Without technology—accessible technology—PAVE would never be able to support those who rely on us for accurate information and resources.” says Barb Koumjian, Project Coordinator for Lifespan Respite WA at PAVE. This includes the highly accessible PAVE website, with links to parent training programs, family health resources, and support systems.

“All of us at PAVE are deeply committed to addressing the concerns of parents worried about their loved one in school, navigating medical supports, or caregiving for a family member. PAVE’s goal is to provide a seamless online experience, allowing everyone to find information quickly, get support, and hopefully get some peace of mind,” adds Communications Specialist Nicol Walsh. “PAVE’s goal is to provide a seamless online experience, allowing everyone to find information quickly and get support.”

PAVE supports accessibility via adaptive technology: “For the families I support at PAVE, there is an uprising of parents advocating for AAC, in any capacity, at an early age with an autism diagnosis,” says Shawnda Hicks, PAVE Coordinator. “Giving children communication in early learning stages reduces frustration and high behaviors.”

Connecting with PAVE

Cute, mixed race child during hearing exam wears special headphones.

Proud to be a UW CREATE Community Partner

“As a statewide organization, we’re deeply committed to accessibility and equity for everyone, and we value our collaborations with UW CREATE for all we serve in Washington,” says Tracy Kahlo, PAVE Executive Director. 


Thanks to these PAVE staff members for contributing words, data, and perspective: Barb Koumjian, Nicol Walsh, Shawnda Hicks, and Tracy Kahlo.

UW News: Can AI help boost accessibility? CREATE researchers tested it for themselves

November 2, 2023 | UW News

Generative artificial intelligence tools like ChatGPT, an AI-powered language tool, and Midjourney, an AI-powered image generator, can potentially assist people with various disabilities. They could summarize content, compose messages, or describe images. Yet they also regularly spout inaccuracies and fail at basic reasoningperpetuating ableist biases.

This year, seven CREATE researchers conducted a three-month autoethnographic study — drawing on their own experiences as people with and without disabilities — to test AI tools’ utility for accessibility. Though researchers found cases in which the tools were helpful, they also found significant problems with AI tools in most use cases, whether they were generating images, writing Slack messages, summarizing writing or trying to improve the accessibility of documents.

Four AI-generated images show different interpretations of a doll-sized “crocheted lavender husky wearing ski goggles,” including two pictured outdoors and one against a white background.

The team presented its findings Oct. 22 at the ASSETS 2023 conference in New York.

“When technology changes rapidly, there’s always a risk that disabled people get left behind,” said senior author Jennifer Mankoff, CREATE’s director and a professor in the Paul G. Allen School of Computer Science & Engineering. “I’m a really strong believer in the value of first-person accounts to help us understand things. Because our group had a large number of folks who could experience AI as disabled people and see what worked and what didn’t, we thought we had a unique opportunity to tell a story and learn about this.”

The group presented its research in seven vignettes, often amalgamating experiences into single accounts to preserve anonymity. For instance, in the first account, “Mia,” who has intermittent brain fog, deployed ChatPDF.com, which summarizes PDFs, to help with work. While the tool was occasionally accurate, it often gave “completely incorrect answers.” In one case, the tool was both inaccurate and ableist, changing a paper’s argument to sound like researchers should talk to caregivers instead of to chronically ill people. “Mia” was able to catch this, since the researcher knew the paper well, but Mankoff said such subtle errors are some of the “most insidious” problems with using AI, since they can easily go unnoticed.

Yet in the same vignette, “Mia” used chatbots to create and format references for a paper they were working on while experiencing brain fog. The AI models still made mistakes, but the technology proved useful in this case.

“When technology changes rapidly, there’s always a risk that disabled people get left behind.”

Jennifer Mankoff, CREATE Director, professor in the Allen School

Mankoff, who’s spoken publicly about having Lyme disease, contributed to this account. “Using AI for this task still required work, but it lessened the cognitive load. By switching from a ‘generation’ task to a ‘verification’ task, I was able to avoid some of the accessibility issues I was facing,” Mankoff said.

The results of the other tests researchers selected were equally mixed:

  • One author, who is autistic, found AI helped to write Slack messages at work without spending too much time troubling over the wording. Peers found the messages “robotic,” yet the tool still made the author feel more confident in these interactions.
  • Three authors tried using AI tools to increase the accessibility of content such as tables for a research paper or a slideshow for a class. The AI programs were able to state accessibility rules but couldn’t apply them consistently when creating content.
  • Image-generating AI tools helped an author with aphantasia (an inability to visualize) interpret imagery from books. Yet when they used the AI tool to create an illustration of “people with a variety of disabilities looking happy but not at a party,” the program could conjure only fraught images of people at a party that included ableist incongruities, such as a disembodied hand resting on a disembodied prosthetic leg.

“I was surprised at just how dramatically the results and outcomes varied, depending on the task,” said lead author Kate Glazko, a UW doctoral student in the Allen School. “”n some cases, such as creating a picture of people with disabilities looking happy, even with specific prompting — can you make it this way? — the results didn’t achieve what the authors wanted.”

The researchers note that more work is needed to develop solutions to problems the study revealed. One particularly complex problem involves developing new ways for people with disabilities to validate the products of AI tools, because in many cases when AI is used for accessibility, either the source document or the AI-generated result is inaccessible. This happened in the ableist summary ChatPDF gave “Mia” and when “Jay,” who is legally blind, used an AI tool to generate code for a data visualization. He could not verify the result himself, but a colleague said it “didn’t make any sense at all.”  The frequency of AI-caused errors, Mankoff said, “makes research into accessible validation especially important.”

Mankoff also plans to research ways to document the kinds of ableism and inaccessibility present in AI-generated content, as well as investigate problems in other areas, such as AI-written code.

“Whenever software engineering practices change, there is a risk that apps and websites become less accessible if good defaults are not in place,” Glazko said. “For example, if AI-generated code were accessible by default, this could help developers to learn about and improve the accessibility of their apps and websites.”

Co-authors on this paper are Momona Yamagami, who completed this research as a UW postdoctoral scholar in the Allen School and is now at Rice University; Aashaka DesaiKelly Avery Mack and Venkatesh Potluri, all UW doctoral students in the Allen School; and Xuhai Xu, who completed this work as a UW doctoral student in the Information School and is now at the Massachusetts Institute of Technology. This research was funded by Meta, Center for Research and Education on Accessible Technology and Experiences (CREATE), Google, an NIDILRR ARRT grant and the National Science Foundation.


For more information, contact Glazko at glazko@cs.washington.edu and Mankoff at jmankoff@cs.washington.edu.


This article was adapted from the UW News article by Stefan Milne.

UW News: A11yBoard accessible presentation software

October 30, 2023 | UW News

A team led by CREATE researchers has created A11yBoard for Google Slides, a browser extension and phone or tablet app that allows blind users to navigate through complex slide layouts, objects, images, and text. Here, a user demonstrates the touchscreen interface. Team members Zhuohao (Jerry) Zhang, Jacob O. Wobbrock, and Gene S-H Kim presented the research at ASSETS 2023.

A user demonstrates creating a presentation slide with A11yBoard on a touchscreen tablet and computer screen.

Screen readers, which convert digital text to audio, can make computers more accessible to many disabled users — including those who are blind, low vision or dyslexic. Yet slideshow software, such as Microsoft PowerPoint and Google Slides, isn’t designed to make screen reader output coherent. Such programs typically rely on Z-order — which follows the way objects are layered on a slide — when a screen reader navigates through the contents. Since the Z-order doesn’t adequately convey how a slide is laid out in two-dimensional space, slideshow software can be inaccessible to people with disabilities.

Combining a desktop computer with a mobile device, A11yBoard lets users work with audio, touch, gesture, speech recognition and search to understand where different objects are located on a slide and move these objects around to create rich layouts. For instance, a user can touch a textbox on the screen, and the screen reader will describe its color and position. Then, using a voice command, the user can shrink that textbox and left-align it with the slide’s title.

“We want to empower people to create their own content, beyond a PowerPoint slide that’s just a title and a text box.”

Jacob O. Wobbrock, CREATE associate director and professor in the UW Information School

“For a long time and even now, accessibility has often been thought of as, ‘We’re doing a good job if we enable blind folks to use modern products.’ Absolutely, that’s a priority,” said senior author Jacob O. Wobbrock, a UW professor in the Information School. “But that is only half of our aim, because that’s only letting blind folks use what others create. We want to empower people to create their own content, beyond a PowerPoint slide that’s just a title and a text box.”

A11yBoard for Google Slides builds on a line of research in Wobbrock’s lab exploring how blind users interact with “artboards” — digital canvases on which users work with objects such as textboxes, shapes, images and diagrams. Slideshow software relies on a series of these artboards. When lead author Zhuohao (Jerry) Zhang, a UW doctoral student in the iSchool, joined Wobbrock’s lab, the two sought a solution to the accessibility flaws in creativity tools, like slideshow software. Drawing on earlier research from Wobbrock’s lab on the problems blind people have using artboards, Wobbrock and Zhang presented a prototype of A11yBoard in April. They then worked to create a solution that’s deployable through existing software, settling on a Google Slides extension.

For the current paper, the researchers worked with co-author Gene S-H Kim, an undergraduate at Stanford University, who is blind, to improve the interface. The team tested it with two other blind users, having them recreate slides. The testers both noted that A11yBoard greatly improved their ability to understand visual content and to create slides themselves without constant back-and-forth iterations with collaborators; they needed to involve a sighted assistant only at the end of the process.

The testers also highlighted spots for improvement: Remaining continuously aware of objects’ positions while trying to edit them still presented a challenge, and users were forced to do each action individually, such as aligning several visual groups from left to right, instead completing these repeated actions in batches. Because of how Google Slides functions, the app’s current version also does not allow users to undo or redo edits across different devices.

Ultimately, the researchers plan to release the app to the public. But first they plan to integrate a large language model, such as GPT, into the program.

“That will potentially help blind people author slides more efficiently, using natural language commands like, ‘Align these five boxes using their left edge,’” Zhang said. “Even as an accessibility researcher, I’m always amazed at how inaccessible these commonplace tools can be. So with A11yBoard we’ve set out to change that.”

This research was funded in part by the University of Washington’s Center for Research and Education on Accessible Technology and Experiences (UW CREATE). For more information, contact Zhang at zhuohao@uw.edu and Wobbrock at wobbrock@uw.edu.


This article was adapted from the UW News article by Stefan Milne.

Augmented Reality to Support Accessibility

October 25, 2023

RASSAR – Room Accessibility and Safety Scan in Augmented Reality – is a novel smartphone-based prototype for semi-automatically identifying, categorizing, and localizing indoor accessibility and safety issues. With RASSAR, the user holds out their phone and scans a space. The tool uses LiDAR and camera data, real-time machine learning, and AR to construct a real-time model of the 3D scene, attempts to identify and classify known accessibility and safety issues, and visualizes potential problems overlaid in AR. 

RASSAR researchers envision the tool as an aid in the building and validation of new construction, planning renovations, or updating homes for health concerns, or for telehealth home visits with occupational therapists. UW News interviewed two CREATE Ph.D. students about their work on the project:


Augmented Reality to Support Accessibility

CREATE students Xia Su and Jae Lee, advised by CREATE Associate Director Jon Froehlich in the Makeability Lab, discuss their work using augmented reality to support accessibility. The Allen School Ph.D. students are presenting their work at ASSETS and UIST this year.

Illustration of a user holding a smartphone using the RASSAR prototype app to scan the room for accessibility issues.

ASSETS 2023 Papers and Posters

October 4, 2023


Augmented Reality to Support Accessibility

CREATE students Xia Su and Jae Lee, advised by CREATE Associate Director Jon Froehlich in the Makeability Lab, discuss their work using augmented reality to support accessibility. The Allen School Ph.D. students are presenting their work at ASSETS and UIST this year.

Illustration of a user holding a smartphone using the RASSAR prototype app to scan the room for accessibility issues.

As has become customary, CREATE faculty, students and alumni will have a large presence at the 2023 ASSETS Conference. It’ll be quiet on campus October 23-25 with these folks in New York.

Papers and presentations

How Do People with Limited Movement Personalize Upper-Body Gestures? Considerations for the Design of Personalized and Accessible Gesture Interfaces
Monday, Oct 23 at 11:10 a.m. Eastern time
Momona Yamagami, Alexandra A Portnova-Fahreeva, Junhan Kong, Jacob O. Wobbrock, Jennifer Mankoff

Understanding Digital Content Creation Needs of Blind and Low Vision People
Monday, Oct 23 at 1:40 p.m. Eastern time
Lotus Zhang, Simon Sun, Leah Findlater

Notably Inaccessible — Data Driven Understanding of Data Science Notebook (In)Accessibility
Monday, Oct 23 at 4 p.m. Eastern time
Venkatesh Potluri, Sudheesh Singanamalla, Nussara Tieanklin, Jennifer Mankoff

A Large-Scale Mixed-Methods Analysis of Blind and Low-vision Research in ACM and IEEE
Tuesday, Oct 24 at 11:10 a.m. Eastern time
Yong-Joon Thoo, Maximiliano Jeanneret Medina, Jon E. Froehlich, Nicolas Ruffieux, Denis Lalanne

Working at the Intersection of Race, Disability and Accessibility
Tuesday, Oct 24 at 1:40 p.m. Eastern time
Christina Harrington, Aashaka Desai, Aaleyah Lewis, Sanika Moharana, Anne Spencer Ross, Jennifer Mankoff

Comparing Locomotion Techniques in Virtual Reality for People with Upper-Body Motor Impairments
Wednesday, Oct 25 at 8:45 a.m. Eastern time
Rachel L. Franz, Jinghan Yu, Jacob O. Wobbrock

Jod: Examining the Design and Implementation of a Videoconferencing Platform for Mixed Hearing Groups
Wednesday, Oct 25 at 11:10 a.m. Eastern time
Anant Mittal, Meghna Gupta, Roshni Poddar, Tarini Naik, SeethaLakshmi Kuppuraj, James Fogarty. Pratyush Kumar, Mohit Jain

Azimuth: Designing Accessible Dashboards for Screen Reader Users
Wednesday, Oct 25 at 1:25 p.m. Eastern time
Arjun Srinivasan, Tim Harshbarger, Darrell Hilliker, Jennifer Mankoff

Developing and Deploying a Real-World Solution for Accessible Slide Reading and Authoring for Blind Users
Wednesday, Oct 25 at 1:25 p.m. Eastern time
Zhuohao Zhang, Gene S-H Kim, Jacob O. Wobbrock

Experience Reports

An Autoethnographic Case Study of Generative Artificial Intelligence’s Utility for Accessibility
Kate S Glazko, Momona Yamagami, Aashaka Desai, Kelly Avery Mack, Venkatesh Potluri, Xuhai Xu, Jennifer Mankoff

Maintaining the Accessibility Ecosystem: a Multi-Stakeholder Analysis of Accessibility in Higher Education
Kelly Avery Mack, Natasha A Sidik, Aashaka Desai, Emma J McDonnell, Kunal Mehta, Christina Zhang, Jennifer Mankoff

TACCESS Papers

“I’m Just Overwhelmed”: Investigating Physical Therapy Accessibility and Technology Interventions for People with Disabilities and/or Chronic Conditions

Momona Yamagami, Kelly Mack, Jennifer Mankoff, Katherine M. Steele

The Global Care Ecosystems of 3D Printed Assistive Devices

Saiph Savage, Claudia Flores-Saviaga, Rachel Rodney, Liliana Savage, Jon Schull, Jennifer Mankoff

Posters

Conveying Uncertainty in Data Visualizations to Screen-Reader Users Through Non-Visual Means
Ather Sharif, Ruican Zhong, Yadi Wang

U.S. Deaf Community Perspectives on Automatic Sign Language Translation
Nina Tran, Richard E. Ladner, Danielle Bragg (Microsoft Research)

Workshops

Bridging the Gap: Towards Advancing Privacy and Accessibility
Rahaf Alharbi, Robin Brewer, Gesu India, Lotus Zhang, Leah Findlater, and Abigale Stangl

Tackling the Lack of a Practical Guide in Disability-Centered Research
Emma McDonnell, Kelly Avery Mack, Kathrin Gerling, Katta Spiel, Cynthia Bennett, Robin N. Brewer, Rua M. Williams, and Garreth W. Tigwell

A11yFutures: Envisioning the Future of Accessibility Research
Foad Hamidi Kirk Crawford, Jason Wiese, Kelly Avery Mack, Jennifer Mankoff

Demos

A Demonstration of RASSAR : Room Accessibility and Safety Scanning in Augmented Reality
Xia Su, Kaiming Cheng, Han Zhang, Jaewook Lee, Wyatt Olson, Jon E. Froehlich

BusStopCV: A Real-time AI Assistant for Labeling Bus Stop Accessibility Features in Streetscape Imagery
Chaitanyashareef Kulkarni, Chu Li, Jaye Ahn, Katrina Oi Yau Ma, Zhihan Zhang, Michael Saugstad, Kevin Wu, Jon E. Froehlich; with Valerie Novack and Brent Chamberlain (Utah State University)

Papers and presentations by CREATE associates and alumni

  • Monday, Oct 23 at 4:00 p.m. Eastern time
    Understanding Challenges and Opportunities in Body Movement Education of People who are Blind or have Low Vision
    Madhuka Thisuri De Silva, Leona M Holloway, Sarah Goodwin, Matthew Butler
  • Tuesday, Oct 24 at 8:45 a.m. Eastern time
    AdaptiveSound: An Interactive Feedback-Loop System to Improve Sound Recognition for Deaf and Hard of Hearing Users
    Hang Do, Quan Dang, Jeremy Zhengqi Huang, Dhruv Jain
  • Tuesday, Oct 24 at 8:45 a.m. Eastern time
    “Not There Yet”: Feasibility and Challenges of Mobile Sound Recognition to Support Deaf and Hard-of-Hearing People
    Jeremy Zhengqi Huang, Hriday Chhabria, Dhruv Jain
  • Tuesday, Oct 24 at 4:00 p.m. Eastern time
    The Potential of a Visual Dialogue Agent In a Tandem Automated Audio Description System for Videos
    Abigale Stangl, Shasta Ihorn, Yue-Ting Siu, Aditya Bodi, Mar Castanon, Lothar D Narins, Ilmi Yoon

Recommended Reading: Parenting with a Disability

October 16, 2023

Two recent publications address unnecessary challenges faced by parents with disabilities and how those challenges are made extraordinary by a legal system that is not protecting parents or their children.

Rocking the Cradle: Ensuring the Rights of Parents with Disabilities and Their Children

The National Council on Disability report finds that roughly 4 million parents in the U.S. who are disabled (about 6% of parents) are the only distinct community that must struggle to retain custody of their children. 

While we have moved (somewhat) beyond the blatant eugenics of the 20th century, some of those tactics persist. Further, “parents with disabilities are the only distinct community of Americans who must struggle to retain custody of their children.” This is also connected to other intersectional factors. For example, “Because children from African American and Native American families are more likely to be poor, they are more likely to be exposed to mandated reporters as they turn to the public social service system for support in times of need…”

Research has shown that exposure bias is evident at each decision point in the child welfare system.

Under the Watchful Eye of All: Disabled Parents and the Family Policing System’s Web of Surveillance

Author Robyn Powel details how the child welfare system employs extensive surveillance that disproportionately targets marginalized families. Yet centers for independent living and other existing programs have the potential to support these parents. Instead, “The child welfare system, more accurately referred to as the family policing system, employs extensive surveillance that disproportionately targets marginalized families, subjecting them to relentless oversight.”

One particular story in that article highlights the role of technology in this ‘policing’: “…just as the Hackneys were preparing to bring [their 8 month old] home, the Allegheny County DHS [alleged] negligence due to [the parents’] disabilities… More than a year later, their toddler remains in the foster care system, an excruciating separation for the Hackneys. The couple is left questioning whether DHS’ use of a predictive artificial intelligence (“AI”) tool unfairly targeted them based on their disabilities.”

As technologists, we wonder whether this AI tool was tested for racial or disability bias. It is essential that the technologies we create are equitable before they are deployed. 

Research at the Intersection of Race, Disability and Accessibility

October 13, 2023

What are the opportunities for research to engage the intersection of race and disability?

What is the value of considering how constructs of race and disability work alongside each other within accessibility research studies?

Two CREATE Ph.D. students have explored these questions and found little focus on this intersection within accessibility research. In their paper, Working at the Intersection of Race, Disability and Accessibility (PDF), they observe that we’re missing out on the full nuance of marginalized and “otherized” groups. 

The Allen School Ph.D. students, Aashaka Desai and Aaleyah Lewis, and collaborators will present their findings at the ASSETS 2023 conference on Tuesday, October 24.

Spurred by the conversation at the Race, Disability & Technology research seminar earlier in the year, members of the team realized they lacked a framework for thinking about work at this intersection. In response, they assembled a larger team to conduct an analysis of existing work and research with accessibility research.

The resulting paper presents a review of considerations for engaging with race and disability in the research and education process. It offers analyses of exemplary papers, highlights opportunities for intersectional engagement, and presents a framework to explore race and disability research. Case studies exemplify engagement at this intersection throughout the course of research, in designs of socio-technical systems, and in education. 


   Case studies

  • Representation in image descriptions: How to describe appearance, factoring preferences for self-descriptions of identity, concerns around misrepresentation by others, interest in knowing others’ appearance, and guidance for AI-generated image descriptions.
  • Experiences of immigrants with disabilities: Cultural barriers that include cultural disconnects and levels of stigma about disability between refugees and host countries compound language barriers.
  • Designing for intersectional, interdependent accessibility: How access practices as well as cultural and racial practices influence every stage of research design, method, and dissemination, in the context of work with communities of translators.

Composite image of the six authors of a variety of backgrounds: Christina Harringon, Aashaka Desai, Aaleyah Lewis, Sanika Moharana, Anned Spencer Ross, and Jennifer Mankoff
Authors, left to right: Christina Harringon, Aashaka Desai, Aaleyah Lewis, Sanika Moharana, Anne Spencer Ross, and Jennifer Mankoff

Authors

CREATE’s Response to Proposed Update to Section 504 for Medical, Health

Updated November 30, 2023

The U.S. Department of Health and Human Services’ (HHS) Office for Civil Rights published a proposed update to the HHS regulations implementing Section 504 of the Rehabilitation Act of 1973, which prohibits disability discrimination by recipients of federal funding.

This is the first comprehensive update to the regulations since they were first put in place more than 40 years ago. The proposed rule includes new requirements prohibiting discrimination in the areas of:

  • Medical treatment
  • The use of value assessments
  • Web, mobile, and kiosk accessibility
  • Requirements for accessible medical equipment, so that persons with disabilities have an opportunity to participate in or benefit from health care programs and activities that is equal to the opportunity afforded others.

For 60 days starting on September 14, HHS will be seeking public comment on the proposed rule. Input from the disability and aging communities is essential!


Note that CREATE also provided a review guide and CREATE’s response in an accessible and tagged PDF document (53 pages) for a previous public comment invitation, specifically for the U.S. Department of Justice in the areas of digital accessibility.

CREATE’s Response to Proposed Digital Accessibility Guidelines

October 4, 2023

CREATE has submitted a response, in collaboration with colleagues within the UW and at peer institutions, to the U.S. Department of Justice (DoJ) proposal for new digital accessibility guidelines for entities that receive federal funds (schools, universities, agencies, etc.). The DoJ proposal invited review of the proposed guidelines.

The response commends the Department of Justice for addressing the issue of inaccessible websites and mobile apps for Title II entities through the approach proposed through the Notice of Proposed Rulemaking (NPRM). The future popularity of websites and apps was not anticipated when the Americans with Disabilities Act was signed into law in 1990. Since then, websites, non-web documents, mobile apps, and other software have become popular ways for Title II entities to reach out and inform the public, to offer benefits and activities, and to use as a part of their offerings to members of the public. In recent years, many entities have asked for clearer legal guidance, so we appreciate the Department’s efforts to address these issues in proposed rulemaking. 

Read more:

Virtual Traffic Stop App Aims to Ease Tensions, Aid Communication

October 2, 2023

The designers of the Virtual Traffic Stop app aim to ease tensions and prevent misunderstandings between drivers and law enforcement during traffic stops. For Hard-of-Hearing or Deaf drivers, the app can be used to communicate with law enforcement via chat during the video. Users can add family members and invite them to the chat for additional assistance.

A Gainesville Florida K-12 school has announced their endorsement of Virtual Traffic Stop and has encouraged parents and their children to sign up and start using the app. Currently, the app is being used by the University of Florida and Gainesville Florida police departments.

If your community is interested in using the app, contact Dr. Juan E. Gilbert, a former CREATE Advisory Board member and Chair of the Human-Centered Computing Department at the University of Florida, by calling 352-392-1527 or emailing juan@ufl.edu.

People with Disabilities are a Population with Health Disparities

September 29, 2023

In September 2023, the Director of the National Institute on Minority Health and Health Disparities announced the designation of people with disabilities as a population with health disparities. The designation is one of several steps National Institutes of Health (NIH) is taking to address health disparities faced by people with disabilities and ensure their representation in NIH research.

Dr. Eliseo J. Pérez-Stable, in consultation with Dr. Robert Otto Valdez, the Director of the Agency for Healthcare Research and Quality cited careful consideration of the National Advisory Council on Minority Health and Health Disparities final report, input from the disability community, and a review of the science and evidence.

As part of the effort to support research in this area, NIMHD also announced a funding opportunity to advance the science of disabilities research.

Read more

Your Review & Comments Wanted: Proposed Federal Accessibility Standards

August 11, 2023

A proposal for new digital accessibility guidelines for entities receiving federal funds was released for review by the U.S. Department of Justice (DOJ) on August 4, 2023.  Anyone affected by these guidelines has 60 days — through Tuesday, October 3, 2023 — to comment.

The DOJ is still trying to decide exactly what the rule should say, how quickly public entities should improve digital accessibility, and what exceptions to allow. For example, the current rule states that course content posted on a password-protected website (such as a learning management system (LMS) like Canvas) does not have to be made accessible until a student with a disability needs access to that content. If a student registers for the course, or transfers into it, then the course content has to be made fully accessible to all disabilities by the start of the term or within 5 days (if the term has already started). In addition, the course needs to stay accessible over time.

CREATE Director Jennifer Mankoff summarizes some of the most important aspects of the proposed rule in a Guide to Reviewing and Commenting that includes many of the the questions posed by the DOJ, with additional questions to consider from Mankoff. This guide is not meant to direct your comments, rather to facilitate and encourage your review. Whatever your viewpoint on the questions raised, the DOJ should hear from you

We strongly urge you to review the guidelines and submit your comments. If you have any questions, reach out to CREATE at create-contact@uw.edu.

Proposed Federal Accessibility Standards: CREATE’s Guide to Reviewing and Commenting

A proposal for new digital accessibility guidelines for entities receiving federal funds was released for review by the U.S. Department of Justice on August 4, 2023. Anyone affected by these guidelines had until October 3, 2023 to comment.

  • CREATE’s official response, in collaboration with colleagues within the UW and at peer institutions, is posted on the DOJ site temporarily.
  • The response is available as an accessible and tagged PDF document (53 pages).
  • If you have any questions, reach out to CREATE at create-contact@uw.edu.

August 2023 announcement

Note that the comment period has ended.

The U.S. Department of Justice (DoJ) is proposing new requirements for digital accessibility for the Americans with Disabilities Act (ADA). Their goal is to provide public entities with clear and concrete standards for how to fulfill their obligations under the Americans with Disabilities Act Title II Regulations. The goal of these new standards is to ensure public entities provide equal access to all services, programs, and activities that are provided via the web and mobile apps. 

These standards impact mobile apps, websites, and course materials created by and for government bodies, including public schools (K-12 and universities), and public services of all kinds.

Below, we have tried to summarize some of the most important aspects of the proposed rule and to explain them. However, in summarizing we have naturally emphasized things we think are important. Some of the topics the rule touches on that we summarize below include the proposed timeline for making digital content accessible; the proposed rules impacting K-12 and college/university course content; what standards should be met for digital content to be accessible for websites, apps, and live audio captioning; and how compliance should be assessed

Note that submitted comments are publicly available online at: DOJ-CRT-2023-0007 on www.regulations.gov.

roposed rule, but the DOJ has asked a number of very specific questions that you might want to comment on.

Notable questions, highlighted

We highlight several of the DOJ questions below, labeled with the Question # that the DOJ uses for them. You will see that we present these questions out of order – we present them in the order that made sense to us when we summarized this proposed rule. You can read the whole proposed rule and all the questions, in order, on the posting of Docket (DOJ-CRT-2023-0007) on www.regulations.gov. Sometimes we write Question to Consider before a question; these are questions we think you might want to comment on even though the DOJ did not ask about them.

Why submit comments?

The DOJ is still trying to decide exactly what the rule should say, how quickly public entities should improve digital accessibility, and what exceptions to allow. For example, the current rule states that course content posted on a password-protected website (such as a learning management system (LMS) like Canvas) does not have to be made accessible until a student with a disability needs access to that content. If a student registers for the course, or transfers into it, then the course content has to be made fully accessible to all disabilities by the start of the term or within 5 days (if the term has already started). In addition, the course needs to stay accessible over time.

If you agree with this, you might want to say so in your comments, because someone else might think this is unreasonable, and the DOJ should hear from both sides. But you might disagree, in which case you should also comment.  

What are the new standards about?

These standards affect web content and mobile apps. These are very broadly defined and include almost any digital content that is important to interacting with public entities. 

Web content is defined as “information or sensory experience that is communicated to the user by a web browser or other software. This includes text, images, sounds, videos, controls, animations, navigation menus, and documents.” It also includes things like web content posted on social media apps, to the extent possible (for example, if the app supports it, the public entity should add image descriptions to images it posts).


Question 1: The DOJ’s definition of “conventional electronic documents” consists of an exhaustive list of specific file types. Should the DOJ instead craft a more flexible definition that generally describes the types of documents that are covered or otherwise change the proposed definition, such as by including other file types (e.g., images or movies), or removing some of the listed file types?


Mobile applications, or “apps,” are defined as “software applications that are designed to be downloaded and run on mobile devices such as smartphones and tablets.” A public entity may use a mobile app that someone else designed and built (an “external mobile app”); in this case it still needs to be accessible. 


Question 25: What types of external mobile apps, if any, do public entities use to offer their services, programs, and activities to members of the public, and how accessible are these apps? … should [these apps be exempt]? If so, should this exception expire after a certain time, and how would this exception impact persons with disabilities.


Timeline for making web content and mobile apps accessible

Almost any content or app that is important to interacting with a public entity has to be made accessible within 2 years of the date when the rule becomes official, regardless of whether a disabled person asks for it or is known to be using that content or app. For public entities that serve a small number of people (<50,000), the proposed deadline is 3 years. For example, a small county police department in a county with <50,000 people would have 3 years. However only truly independent entities qualify for this exception. For example, the same policy department in a county with >50,000 people would only have two years; similarly a small public school in a large county would only have two years.

If a public entity feels this would be too costly, under the proposed rule they must prove this and they still have to do as much as possible, “to the maximum extent possible” to support their disabled constituents.


Question 4: What compliance costs and challenges might small public entities face in conforming with this rule? … [do they have internal staff for addressing accessibility? If they have recently addressed accessibility, how much did that cost?]

Question 5: Should the DOJ adopt a different WCAG version or conformance level for small entities or a subset of small entities?


Exemptions

The new rule does include some exceptions, meaning it allows some content to be inaccessible. However, when a disabled person needs the inaccessible content, existing regulations implementing title II of the ADA may come into effect, typically requiring the content to be made accessible.

Exemptions

The new rule does include some exceptions, meaning it allows some content to be inaccessible. However, when a disabled person needs the inaccessible content, existing regulations implementing title II of the ADA may come into effect, typically requiring the content to be made accessible.

Archived and pre-existing non-web documents

The following exemptions are intended to reduce the burden of the new rule for large collections of rarely used documents:

  • Archived content not in active use
  • Pre-existing non-web documents

The DOJ has several questions about these exemptions (see Questions 15, 16, 17, 18, 19, and 20) which relate to how such content is currently used, where it is posted, and how these exemptions would impact people with disabilities.


Course content

Generally speaking, course content (such as a public syllabus or handout) has to be made accessible. However, course content inside a password protected website such as a learning management system (for both K-12 schools and colleges/universities) is exempted if the content is only available specifically to admitted students enrolled in the relevant course (and disabled parents, in the case of K-12 materials). 

Once an institution knows, or should have known that a student (or, for K-12 courses, parent) with a disability is enrolled in the course, “all existing course content must be made fully accessible by the [start of the academic term] for that course… New content added throughout the term for the course must also comply… at the time it is added to the website.”

Under today’s interpretation of the ADA, transferring to a course during the add/drop period or from a waitlist often means that the course is less accessible. The DOJ guidelines address this, requiring in these cases that course material is made accessible within five business days of the student’s enrollment. The DOJ also requires “auxiliary aids and services… that enable the student with a disability to participate” while a course is being made accessible. Notably, the relevant material would need to be fully accessible to all disabilities, “not merely the criteria related to that student or parent’s disability.

Importantly, the obligation to make the course content accessible is “ongoing for the duration of the course” and “as long as that content is available to students on the password-protected course website.” It is not clear whether this applies to future offerings of the same course, as typically use of an LMS involves creating a “new” password-protected site for each offering.

The DOJ has a lengthy analysis of tangible and intangible benefits of this ruling, as well as expected costs. They estimate that “By the end of year four (two years after postsecondary schools begin to remediate course content), 96 percent of courses offered by public four-year and postgraduate institutions and 90 percent of courses offered by community colleges will have been remediated. They further estimate that postsecondary institutions will finish remediation on their own to preempt requests in the following year.” They have similar estimates about K-12 education.


Question to consider: Should the “the duration of a course” apply to a single offering of a course for a single term, or all offerings of that course in all terms (even if a separate LMS site, with a separate password, was created for each offering)? How might this impact the likelihood that most courses are fully accessible within four years?

Question to Consider: Do  you agree that the proposed rule will increase course accessibility to include 96% of courses? How might the variable representation of people with disabilities across fields impact this? For example, people with disabilities are particularly under-represented in STEM fields, where diagrams and math equations are often particularly inaccessible. What could be changed about the proposed rule to make this prediction more likely to come true?


The DOJ also has several questions; we highlight some of them below. We have combined the questions for K-12 and post-secondary educational institutions by referring to [public educational institutions] since the primary difference is that parents with disabilities trigger the need to make documents in the case of K-12 education only. 


Question 27 & 36: How difficult would it be for [public educational institutions] to comply with this rule in the absence of this exception?

Question 28 & 37: What would the impact of this exception be on people with disabilities?

Question 33 & 42: How long would it take to make course content available on a public entity’s password-protected or otherwise secured website for a particular course accessible, and does this vary based on the type of course? Do students need access to course content before the first day of class? How much delay in accessing online course content can a student reasonably overcome in order to have an equal opportunity to succeed in a course, and does the answer change depending on the point in the academic term that the delay occurs?

Question 35 & 44: Should the DOJ consider an alternative approach, such as requiring that all newly posted course content be made accessible on an expedited time frame, while adopting a later compliance date for remediating existing content?


This includes third party content. For example, a website for practicing math problems provided, if required to complete coursework, would need to be accessible. The rules do not specifically mention textbooks. However the DOJ asks:


Question 26: Are there particular issues relating to the accessibility of digital books and textbooks that the DOJ should consider in finalizing this rule? Are there particular issues that the DOJ should consider regarding the impact of this rule on libraries?

Question to Consider: Has textbook accessibility been a barrier to accessing courses? What are some examples of problems you’ve encountered? How common are these problems? What could help?


Other exemptions

The DOJ also exempts linked 3rd party information (if it is not providing a direct service) and individualized, password-protected documents (such as personal utility bills). However it specifies that if these documents have deadlines associated with them, and are not accessible, they need to adjust their deadlines “to ensure that a person with a disability has equal access to its services, programs, or activities.” The DOJ asks about whether proper processes are in place:


Question to Consider: How might a delay in receiving an accessible document affect you? For example, could it affect whether you receive care services, money for food, or healthcare services that could cause harm if delayed? If you think this is a concern, what would be a reasonable deadline for receiving these documents?

Question 46: Do public entities have adequate systems for receiving notification that an individual with a disability requires access to an individualized, password-protected conventional electronic document? What kinds of burdens do these notification systems place on individuals with disabilities and how easy are these systems to access? Should the DOJ consider requiring a particular system for notification or a particular process or timeline that entities must follow when they are on notice that an individual with a disability requires access to such a document?


How is “accessible” defined?

The ADA has always included digital accessibility. However, a lack of specific standards in the past has left public entities to define for themselves what compliance looks like. The result has been a lack of consistent attention to accessibility. According to the DOJ, “voluntary compliance … has been insufficient in providing access.”

Now, the DOJ is requiring public entities to follow the Web Content Accessibility Guidelines (WCAG)version 2.1, at the AA level. This is a carefully tested web standard which has recently been expanded to touch on mobile accessibility needs as well. 


Question 3: Are there technical standards or performance standards other than WCAG 2.1 that the Department should consider? … If so, what is a reasonable time frame for State and local compliance with WCAG 2.2 and why? Is there any other standard that the Department should consider, especially in light of the rapid pace at which technology changes?


The DOJ notes that “the Access Board’s section 508 standards include additional requirements applicable to mobile apps that are not in WCAG 2.1 [including]: interoperability requirements to ensure that a mobile app does not disrupt a device’s assistive technology for persons with disabilities (e.g., screen readers for persons who are blind or have low vision); requirements for mobile apps to follow preferences on a user’s phone such as settings for color, contrast, and font size; and requirements for caption controls and audio description controls that enable users to adjust caption and audio description functions.


Question 8: Is WCAG 2.1 Level AA the appropriate accessibility standard for mobile apps? Should the Department instead adopt another accessibility standard or alternative for mobile apps, such as the requirements from section 508 discussed above?


The DOJ also notes that this includes captioning of “live audio,” such as in real-time presentations. They note that many meetings have moved online since the start of the COVID-19 pandemic, making live audio captioning “even more critical for individuals with certain types of disabilities to participate fully in civic life.” Proper live audio captioning includes speaker identification as well as accurate transcription of spoken text, sound effects, and other significant audio. Live audio captioning of this sort cannot be automated, and the DOJ is concerned about costs. They ask:


Question 13: Should the Department consider a different compliance date for the captioning of live-audio content in synchronized media or exclude some public entities from the requirement?

Question 14: What types of live-audio content do public entities and small public entities post? What has been the cost for providing live-audio captioning?


Finally, the DOJ notes that “WCAG 2.1 can be interpreted to permit the development of two separate websites—one for individuals with relevant disabilities and another for individuals without relevant disabilities—even when doing so is unnecessary and when users with disabilities would have a better experience using the main web page.” They rightly point out that this raises “concerns about user experience, segregation of users with disabilities, unequal access to information, and maintenance”. Thus the proposed rule explicitly states that parallel development of a separate website, document or app is “permissible only where it is not possible to make websites and web content directly accessible due to technical limitations (e.g., technology is not yet capable of being made accessible) or legal limitations (e.g., web content is protected by copyright).”  They go on to ask:


Question 49: Would allowing [a separate alternate version of a website, document, or app] due to technical or legal limitations result in individuals with disabilities receiving unequal access to a public entity’s services, programs, and activities?


How will compliance be measured?

The DOJ has many questions about the best way to measure compliance. The DOJ acknowledges that a public entity might reasonably not be in full compliance with all of WCAG 2.1’s AA standards at all times. This is because web content changes frequently, assessments may not always agree, and may include thousands of pages of content, making compliance more difficult than ensuring access to, say, “a town hall that is renovated once a decade…. The Department also believes that slight deviations from WCAG 2.1 Level AA may be more likely to occur without having a detrimental impact on access than is the case with the ADA Standards. Additionally, it may be easier for an aggrieved individual to find evidence of noncompliance with WCAG 2.1 Level AA than noncompliance with the ADA Standards, given the availability of many free testing tools and the fact that public entities’ websites can be accessed from almost anywhere.” 

They discuss several alternatives that could allow for the necessity of slight deviations and short periods of noncompliance while still promoting high compliance overall, including a percentage-based standard (which may be difficult to implement, and may need to weight different aspects of WCAG 2.1 differently to achieve equity); a standard based on policies for feedback, testing and remediation (which may be inconsistently applied); or “organizational maturity” meaning the organization can show it has a robust accessibility program in place (which may not translate to full accessibility or compliance). They solicit commentary on compliance: 


The DOJ asks what evidence an allegation of noncompliance requires (Question 50); Whether organizational feedback practices, testing policies, remediation practices, or organizational maturity should matter in assessing compliance (Questions 51, 55, 58). The DOJ also asks about what specific feedback practices, testing policies, remediation policies, and level of organizational maturity are needed (Questions 52, 53, 54, 59). They also ask:

Question 62: Should the Department address the different level of impact that different instances of nonconformance with a technical standard might have on the ability of people with disabilities to access the services, programs, and activities that a public entity offers via the web or a mobile app? If so, how?


To conclude, the DOJ’s proposed rule covers a number of topics that are of great importance to people with disabilities. We strongly urge you to comment on the rule.

If you have any questions, reach out to CREATE at create-contact@uw.edu.

CREATE Open Source Projects Awarded at Web4All

July 6, 2023

CREATE researchers shone this spring at the 2023 Web4All 2023 conference that, in part, seeks to “make the internet more accessible to the more than one billion people who struggle to interact with digital content each day due to neurodivergence, disability or other impairments.” Two CREATE-funded open source projects won accolades.

Best Technical Paper award:
Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users

Authors: Ather Sharif, Andrew Mingwei Zhang, CREATE faculty member Katharina Reinecke, and CREATE Associate Director Jacob O. Wobbrock

Built on prior research to develop taxonomies of information sought by screen-reader users to interact with online data visualizations, the team’s research used these taxonomies to extend the functionality of VoxLens—an open-source multi-modal system that improves the accessibility of data visualizations—by supporting drilled-down information extraction. They assessed the performance of their VoxLens enhancements through task-based user studies with 10 screen-reader and 10 non-screen-reader users. Their enhancements “closed the gap” between the two groups by enabling screen-reader users to extract information with approximately the same accuracy as non-screen-reader users, reducing interaction time by 22% in the process.

Accessibility Challenge Delegates’ Award:
UnlockedMaps: A Web-Based Map for Visualizing the Real-Time Accessibility of Urban Rail Transit Stations

Authors: Ather Sharif, Aneesha Ramesh, Qianqian Yu, Trung-Anh H. Nguyen, and Xuhai Xu

Ather Sharif’s work on another project, UnlockedMaps, was honored with the Accessibility Challenge Delegates’ Award. The paper details a web-based map that allows users to see in real time how accessible rail transit stations are in six North American cities, including Seattle, Toronto, New York and the Bay Area. UnlockedMaps shows whether stations are accessible and if they are currently experiencing elevator outages. Their work includes a public website that enables users to make informed decisions regarding their commute and an open source API that can be used by developers, disability advocates, and policy makers for a variety of purposes, including shedding light on the frequency of elevator outages and their repair times to identify the disparities between neighborhoods in a given city.

Read more

Accessible Technology Research Showcase – Spring 2023

June 30, 2023

Poster session in progress, with 9 or so posters on easels in view and student presenters talking to attendees.

In June 2023, CREATE and HuskyADAPT co-hosted a showcase — and celebration — of outstanding UW research on accessible technology. The showcase featured poster presentations, live demonstrations by our faculty, students, and researchers and was altogether vibrant and exciting. Over 100 attendees viewed 25 projects, presentations, and posters.

Congratulations and appreciation to CREATE Engagement and Partnerships Manager Kathleen Quin Voss and HuskyAdapt Student Executive Chair Mia Hoffman for putting on an amazing research showcase!

View the Projects


Deep Gratitude to Wobbrock, Ladner & Caspi

June 13, 2023

The CREATE community thanks three of our founding leaders for their energy and service in launching the center as we embark upon some transitions. “CREATE would not be where it is today without the vision, passion, and commitment that Jake, Richard, and Anat brought to their work leading the center,” says CREATE Director Jennifer Mankoff.

Co-Director Jacob O. Wobbrock: From vision, to launch, to sustainable leadership

Jacob O. Wobbrock, a 40-something white man with short hair, a beard, and glasses. He is smiling in front of a white board.

It was back in June 2019 that Jacob O. Wobbrock, CREATE’s founding Co-Director, was on a panel discussion at Microsoft’s IdeaGen 2030 event, where he talked about ability-based design. Also on that panel was future CREATE Associate Director Kat Steele. After the event, the two talked with Microsoft Research colleagues, particularly Dr. Meredith Ringel Morris, about the possibility of founding an accessible technology research center at the University of Washington.

Wobbrock and Steele thought that a center could bring faculty together and make them more than the sum of their parts. Within a few months, Wobbrock returned to Microsoft with Jennifer Mankoff, Richard Ladner, and Anat Caspi to pitch Microsoft’s Chief Accessibility Officer, Jenny Lay-Flurrie, on the idea of supporting the new Center for Research and Education on Accessible Technology and Experiences (CREATE). With additional support from Microsoft President Brad Smith, and input from Morris, the center was launched by Smith and UW President Ana Marie Cauce at Microsoft’s Ability Summit in Spring 2020.

Wobbrock, along with Mankoff, served as CREATE’s inaugural co-directors until June 2023, when Wobbrock stepped down into an associate director role, with Mankoff leading CREATE as sole Director. “I’m a founder by nature,” Wobbrock said. “I helped start DUB, the MHCI+D degree, a startup called AnswerDash, and then CREATE. I really enjoy establishing new organizations and seeing them take flight. Now that CREATE is soaring, it’s time for more capable hands than mine to pilot the plane. Jennifer Mankoff is one of the best, most capable, energetic, and visionary leaders I know. She will take CREATE into its next chapter and I can’t wait to see what she does.” Wobbrock will still be very active with the center.

Professor Emeritus Richard Ladner, one of CREATE’s founders and our inaugural Education Director

Headshot of Richard Ladner. He has grey hair and beard and is wearing a blue shirt and colorful tie.

We thank Professor Emeritus Richard Ladner for three years of leadership as one of our founders and CREATE’s inaugural Education Director. Ladner initiated the CREATE Student Minigrant Program that helps fund small grants up to $2,000 in support of student initiated research projects.

Ladner has shepherded 10 minigrants and worked directly with eight Teach Access Study Away students. Through his AccessComputing program, he helped fund several summer research internships for undergraduate students working with CREATE faculty. All CREATE faculty contribute to accessibility related education in their courses, where he provides encouragement.

Anat Caspi, inaugural Director of Translation

Anat Caspi: A white woman smiling into the camera. She is wearing a purple blouse.

Anat Caspi defined and elevated CREATE’s translation efforts, leveraging the center’s relationships with partners in industry, disability communities, and academia. Her leadership created sustainable models for translation and built on our prior successes. Collaborations with the TASKAR centerHuskyADAPT, and the UW Disability Studies Program have ensured diverse voices to inform innovation. 

Director of Translation duties will be distributed across Mankoff, CREATE’s Community Engagement and Partnerships Manager Kathleen Quin Voss, and the Taskar Center for Accessible Technology, which Caspi directs.

Codesigning Videoconferencing Tools for Small Groups with Mixed Hearing Status

June 12, 2023

CREATE students and faculty have published a new paper at CHI 2023, ‘Easier or Harder, Depending on Who the Hearing Person Is’: Codesigning Videoconferencing Tools for Small Groups with Mixed Hearing Status”.

Led by Human Centered Design and Engineering (HCDE) Ph.D. candidate Emma McDonnell and supported by CREATE, this work investigates how groups with both hearing and d/Deaf and hard of hearing (DHH) members could be better supported when using captions during videoconferences. 

Emma McDonnell, a white woman in her 20s with short red hair, freckles, and a warm smile. In the background: a lush landscape and the Colosseum.

Researchers recruited four groups to participate in a series of codesign sessions, which de-centers researchers’ priorities and seeks to empower participants to lead the development of new design ideas. In the study, participants reflected on their experiences using captioning, sketched and discussed their ideas for technology that could help build accessible group norms, and then critiqued video prototypes researchers created of their ideas. 

One major finding from this research is that participants’ relationships with each other shape what kinds of accessibility support the group would benefit from.

For example, one group that participated in our study were cousins who had been close since childhood. Now in their mid-twenties, they found they did not have to actively plan for accessibility; they had their ways of communicating and would stop and clarify if things broke down. On the other hand, a group of colleagues who work on technology for DHH people had many explicit norms they used to ensure communication accessibility. One participant, Blake, noted, I was pretty emotional after the first meeting because it was just so inclusive.” These different approaches demonstrate that there is no one-size-fits-all approach to communication accessibility – people work together as a group to develop an approach that works for them. 

This paper also contributes new priorities for the design of videoconferencing software. Participants focused on designing add-ons to videoconferencing systems that would better support their group in communicating accessibly. Their designs fell into four categories: 

  • Speaker Identity and Overlap: Having video conferencing tools identify speakers and warn groups when multiple people speak at once, since overlapping speech can’t be captioned accurately. Participants found this to be critical, and often missing, information.
  • Support for Behavioral Feedback: Building in ways for people to subtly notify conversation partners if they need to adjust their behavior. Participants desired tools to flag when people need to adjust their cameras, critical caption errors, and if speech rate gets too high. They considered, but decided against, a general purpose conversation breakdown warning. 
  • Videoconferencing Infrastructure for Accessibility: Adding more features and configurable settings around conversational accessibility to videoconferencing platforms. Participants desired basic controls, such as color and font size, as well as the ability to preset and share group accessibility norms and customize behavior feedback tools. 
  • Sound Information: Providing more information about the sound happening during a conversation. Participants were excited about building sound recognition into captioning tools, and considered conveying speech volume via font weight, but decided it would be overwhelming and ambiguous. 

This research also has implications for broader captioning and videoconferencing design. While often captioning tools are designed for individual d/Deaf and hard of hearing people, researchers argue that we should design for the entire group having a conversation. This shift in focus revealed many ways that, on top of transcribing a conversation, technology could help groups communicate in ways that can be more effectively captioned. Many of these tools are easy to build with current technology, such as being able to click on a confusing caption to request clarification. The research team hopes that their work can illuminate the need to pay attention to groups’ social context when studying captioning and can provide videoconferencing platform designers a design approach to better support groups with mixed hearing abilities. 

McDonnell is advised by CREATE Associate Directors Leah Findlater, HCDE, and Jon Froehlich, Paul G. Allen School of Computer Science & Engineering.

Grant Opportunity: Disability Inclusion

June 11, 2023

The U.S. Department of Labor has made available $2 million for the first year of a cooperative agreement for an employer-focused, disability policy development and technical assistance center. 

From the EARN announcement website:

The purpose of this program is to identify and promote adoption of innovative and equitable evidence-based policy and practice solutions to help public and private sector employers of all sizes recruit, hire, retain, and advance people with disabilities, including those from historically underserved communities.

The entity awarded the EARN cooperative agreement will conduct research that values the perspectives of historically underserved groups, conduct policy analysis to identify and validate effective disability-inclusive policy and practice models, translate that knowledge into engaging tools for employers and intermediary organizations, and provide technical assistance and training to help employers of all sizes, both public and private, create inclusive workplace cultures that support high-quality employment of people with disabilities.