Congratulations to Zhuohao (Jerry) Zhang – the most recent CREATE Ph.D. student to receive an Apple Scholars in AIML PhD fellowship. The prestigious award supports students through funding, internship opportunities, and mentorship with an Apple researcher.
Zhang is a 3rd-year iSchool Ph.D. student advised by Prof. Jacob. O Wobbrock. His research focuses on using human-AI interactions to address real-world accessibility problems. He is particularly interested in designing and evaluating intelligent assistive technologies to make creativity tasks accessible.
Zhang joins previous CREATE-advised Apple AIML fellows:
Venkatesh Potluri (Apple AIML Ph.D. fellow 2022), advised by CREATE Director Jennifer Mankoff in the Allen School. His research makes overlooked software engineering spaces such as IOT and user interface development accessible to developers who are blind or visually impaired. His work systematically understands the accessibility gaps in these spaces and addresses them by enhancing widely used programming tools.
Rachel Franz (Apple AIML Ph.D. fellow 2021) is also advised by Wobbrock in the iSchool. Her research focuses on accessible technology design and evaluation for users with functional impairments and low digital literacy. Specifically, she is focused on using AI to make virtual reality more accessible to individuals with mobility limitations.
A new, free, and community-sourced online book helps Computer Science educators integrate accessibility topics into their classes. Teaching Accessibility provides the foundations of accessibility relevant to computer science teaching and then presents teaching methods for integrating those topics into course designs.
The editors are Alannah Oleson, a postdoctoral scholar and co-founder at the UW Center for Learning, Computing, and Imagination (LCI), CREATE and iSchool faculty Amy Ko, and Richard Ladner, CREATE Director of Education Emeritus. You may recognize many CREATE faculty members’ research referenced throughout the guide. CREATE Director Jennifer Mankoff and CREATE Ph.D. student Avery Kelly Mack contributed a foundational chapter that advocates for teaching inclusively in addition to teaching about accessibility.
Letting the book speak for itself
“… we’ve designed this book as a free, open, living, web-first document. It’s free thanks to a National Science Foundation grant (NSF No. 2137312) that has funded our time to edit and publish the book. It’s open in that you can see and comment on the book at any time, creating community around its content. It’s living in that we expect it to regularly change and evolve as the community of people integrating accessibility into their CS courses grows and evolves. And it’s web-first in that the book is designed first and foremost as an accessible website to be read on desktops, laptops, and mobile devices, rather than as a print book or PDF. This ensures that everyone can read it, but also that it can be easily changed and updated as our understandings of how to teach accessibility in CS evolve.”
Introduction by Alannah Oleson, Amy J. Ko, Richard Ladner
“To write these chapters, we recruited some of the world’s experts on accessible computing and teaching accessible computing, giving them a platform to share both their content knowledge about how accessibility intersects with specific CS topics, but also their pedagogical content knowledge about how to teach those intersections in CS courses.”
Introduction by Alannah Oleson, Amy J. Ko, Richard Ladner
Para.chi is a worldwide parallel event to CHI ’24 for those unable or unwilling to join CHI ‘24. UW Design. Use. Build. (DUB) is hosting para.chi.dub with members of the DUB team–and maybe you.
Live session for accepted virtual papers
Networking opportunities
Accessibility for students and early career researchers locally and online
Wednesday, May 8, 2024 Hybrid event: Seattle location to be announced and virtual info shared upon registration Presenter applications due March 15 Register to attend by Monday, April 1.
Do you have a virtual paper and wish to get feedback from a live audience? Perhaps you have a journal paper accepted to an HCI venue and wish to present it live? Then consider joining us!
Note that presenter space is somewhat limited. Decisions about how to distribute poster, presenter, and hybrid opportunities will be made after March 15.
Seattle and beyond
Each regional team is offering a different event, from mini-conferences to virtual paper sessions to mentoring and networking events.
Completing graduate school is difficult for any student, but it’s especially difficult when you’re trying to learn at an institution that isn’t built for you. Students with disabilities at UW face extra challenges every day because our university doesn’t support equitable participation in educational activities like research and mentorship – those of us who don’t fit the mold face an uphill struggle to make ourselves heard in an academic culture that values maximum efficiency over unique perspectives. In this article, we share three common myths about students with disabilities, reveal the reality of our inequitable experience as grad students at UW, and propose a few potential solutions to begin ameliorating this reality, both at our university and beyond.
Myth 1: DRS (Disabilities Resources for Students) handles all accessibility accommodations.
This is an incorrect expectation of the role DRS serves in a campus ecosystem. The term “accommodations,” in the first place, frames us as outcasts, implying that someone needs to “review” and “approve” of our “requests” to simply exist equitably; but given that this is the term folks are most familiar with, we’ll continue referring to them as “accommodations” for ease of communication. While DRS can provide some assistance, they are outrageously under-staffed, and UW research has demonstrated that they are only part of the ecosystem. Instructors need to consider accessibility when building their courses and when teaching their classes. Accessibility, like computer security, works best when it is considered from the beginning, but it’s not too late to start repairing inaccessible PDFs or lecture slides for a future quarter. UW DO-IT has a great resource for accessible teaching.
Myth 2: Making my materials accessible is all I have to do for disabled students, right?
Disability is highly individual, and no matter how much an instructor prepares, a student might need further accommodations than what was prepared ahead of time. Listen to and believe disabled students when they discuss the accessibility barriers they face. Questioning their disability or using language that makes them doubt their self-worth is a hard no. Then, work with the student to decide on a solution moving forward, and remember that students are the number-one experts on their own accessibility needs.
Myth 3: Advising a student with a disability is the same as advising a student without a disability.
Disabled students have very different experiences of grad school, and they need advisors who are informed, aware, and proactive about those differences. If you are taking on a disabled student, the best ways to prepare yourself are:
Educate yourself about disability.
Disabled students are tired of explaining the same basic accessibility practices over and over again. Be willing to listen if your student wants to educate you more about their experience with disability, and recognize action items from the conversation that you can incorporate to improve your methods.
Expect that timelines might look different.
Disabled students deal with all kinds of barriers, from inaccessible technology to multiple-week hospital stays, so they may do things faster or slower than other students (as is true for any student). This does not mean they are not as productive or deserving of research positions. Disabled students produce high-quality research and award-winning papers, and their unique perspectives have the potential to strengthen every field, not just those related to disability studies. And they are able to do their best work when they have an advisor who recognizes their intellectual merit and right to be a part of the program.
Be prepared to be your student’s number-one ally.
Since DRS cannot fulfill all accessibility needs, you might need to figure out how to solve them yourselves. Can you find $200 in a grant to purchase an OCR tool to help make PDFs accessible for a blind student? (Yes, you can.) Can you advocate for them if their instructor isn’t meeting accessibility requests? (Yes, you can.) Not only will this help them do their best work, but it also sets an example for the other students in your lab and establishes an academic culture that values students of all abilities.
Formally, Dr. Joshua Miele describes himself as a blind scientist, designer, performance artist and disability activist who is focused on the overlap of technology, disability, and equity. But in his personable and humorous lecture, he listed a few more identities: Interrupter. Pain in the ass. “CAOS” promoter.
Miele’s passions are right in line with CREATE’s work and he started his lecture, after being introduced by CREATE Director Jennifer Mankoff, with a compliment we heartily accept: “This community at the University of Washington is one of the largest, one of the most vibrant communities of people thinking and working around disability, accessibility, and technology.”
Miele shared his enthusiasm for disability-inclusive design and its impact on global disability equity and inclusion. Drawing on examples and counterexamples from his own life and career, Dr. Miele described some of the friction the accessibility field has faced and speculated about what challenges may lie ahead, with particular emphasis on the centrality of user-centered practices, and the exhilarating potential of open source solutions and communities.
When he received the MacArthur grant, Miele had to decide what to do with the spotlight on his work. He shared his hopes for a Center for Accessibility and Open Source (CAOS, pronounced “chaos”) to promote global digital equity for people with disabilities through making low-cost accessible tools available to everyone, whether they have financial resources or not. He invited anyone interested in global equity, disability, direct action, performance art, and CAOS/chaos to reach out to work together on this incredibly important work.
CREATE Director Jennifer Mankoff started the conversation asking Wong about her experience as a disabled person in academia and what needs to change. Wong said her work in disability justice was inspired in part by the “incredible amount of emotion and physical labor to ask for equal access” in academic settings. She had to spend precious time, money and energy to gain the accommodations and access she needed to succeed. But she realized that as soon as she transitioned out, her efforts would be lost and the next student would have to start over to prove their need and request a new set of accommodations. Wong was doubtful that large academic institutions can support the goal of collective liberation. It’s the “dog-eat-dog world [of] academia where the competition is stiff and everyone is pushed to their limits to produce and be valuable.” She encouraged instructors to incorporate books about disability justice in their syllabi (see the reading list below).
Wong, who spoke with a text-to-voice tool and added emphasis with her facial expressions on the screen, also addressed the value and the limitations of assistive technology. She noted that the text-to-speech app she uses does not convey her personality. She also discussed how ableism appears in activist discourse.
One of her examples was a debate over gig economy delivery services, which are enormously important for many people with disabilities and that also under-compensate delivery work. She noted that blaming disabled people for undermining efforts for better wages was not the solution; collective efforts to make corporations compensate workers is the solution. She also explained that hashtag activism, which has been disparaged in popular discourse, is a crucial method for disabled people to participate in social justice activism. And she discussed her outrage when, as she prepared to give a talk to a public health school, her own access needs were used to censor her. Throughout her talk, Wong returned again and again to the principles of disability justice, and encouraged attendees to engage in collective forms of change.
Wong’s responses embodied a key component of disability justice principles: citational practices that name fellow contributors to collective disability justice wisdom. Her long list of recommended reading for the audience inspired us to build our new RDT reading list. Wong referenced Patty Berne several times, calling Berne her introduction to disability justice.
Patty Berne on disability justice: Centering intersectionality and liberation
A week later, two CREATE Ph.D. students, Aashaka Desai and Aaleyah Lewis, moderated a conversation with Patty Berne. Berne, who identifies as a Japanese-Haitian queer disabled woman, co-founded Sins Invalid, a disability justice-based arts project focusing on disabled artists of color and queer and gender non-conforming artists with disabilities. Berne defined disability justice as advocating for each other, understanding access needs, and normalizing those needs. On the topic of climate justice, she noted that state-sponsored disaster planning often overlooks the needs of people with motor impairments or life-sustaining medical equipment. This is where intersectional communities do, and should, take care of each other when disaster strikes.
Berne addressed language justice within the disability community, noting that “we don’t ‘language’ like able-bodied people.” For example, the use of ventilators and augmented speech technology change the cadence of speech. Berne wants to normalize access needs for a more inclusive experience of everyday life. Watch the full conversation on YouTube.
Congratulations to Anat Caspi on receiving the 2023 Human Rights Education Award from the Seattle Human Rights Commission!
Caspi, a CREATE associate director and the founder and director of the Taskar Center for Accessible Technology, thanked the commission for recognition of her individual dedication and emphasized that it also celebrates the collective efforts of the Taskar Center community.
In her role as CREATE’s Director of Strategy and Operations, Olivia Banner, Ph.D., will help develop and oversee organizational strategy, design and implement new programs, manage center operations, and help ensure a sustainable trajectory of high quality work in service of the CREATE’s core mission.
Banner is a disabled author and educator who has taught courses on disability, technology, and media. She comes to Seattle and the UW from the University of Texas at Dallas, where she was an associate professor of Critical Media Studies. She is the author of Communicative Biocapitalism: The Voice of the Patient in Digital Health and the Health Humanities. Her new book about technology, psychiatry, and practices of mutual care is forthcoming with Duke University Press. Her research has been published in Catalyst: Feminism, Theory, Technoscience, Literature and Medicine and is forthcoming in Disability Studies Quarterly.
“Her principles and commitment to intersectional work caught our attention in our conversations about the new role. We are so lucky to have her joining CREATE!”
Jennifer Mankoff, CREATE Director
Banner says she looks forward to integrating disability principles into projects with tangible effects on disabled peoples’ lives, including AI + Accessibility, integrating disabled perspectives into projects, and race, technology, and disability—which align with Banner’s previous academic work. She is personally invested in fostering just technological futures through collaborative work and is very excited about the Center’s aim of expanding access through community partnerships.
CREATE Director Jennifer Mankoff is equally excited about the vision Banner brings for CREATE’s future, her policy experience, her administrative skills, and her commitment to amplifying the voices of those she serves. “Her principles and commitment to intersectional work caught our attention in our conversations about the new role. We are so lucky to have her joining CREATE!” says Mankoff.
In her research, scholarship, and teaching, Banner has centered disability knowledge as a method for envisioning technological futures. Her work extends to multiple collaborative projects, including co-teaching a seminar on surveillance with a computer science professor, serving on a Lancet-sponsored commission developing policies for global digital health development, and co-directing Social Practice & Community Engagement Media, a lab that used low-tech methods to reimagine campus practices of care. Toward the goal of improving access on the UT Dallas campus, Banner conducted critical access mapping projects, led Teach-Ins and workshops on disability and equity and on accessible course design, and served on the University Accessibility Committee.
Having served as managing editor of an academic journal and as Associate Dean of Graduate Studies for her School, she also brings professional experience working with faculty, students, staff, and community members from varied disciplines and professions, and anticipates generative conversations on the horizon. She joins CREATE eager to support and enhance its visions of accessible and equitable technology.
People with low vision (LV) have had fewer options for physical activity, particularly in competitive sports such as tennis and soccer that involve fast, continuously moving elements such as balls and players. A group of researchers from CREATE associate director Jon E. Froehlich‘s Makeability Lab hopes to overcome this challenge by enabling LV individuals to participate in ball-based sports using real-time computer vision (CV) and wearable augmented reality (AR) headsets. Their initial focus has been on tennis.
ARTennis is their prototype system capable of tracking and enhancing the visual saliency of tennis balls from a first-person point-of-view (POV). Recent advancements in deep learning have led to models like TrackNet, a neural network capable of tracking tennis balls in third-person recordings of tennis games that is used to improve sports viewing for LV people. To enhance playability, the team first built a dataset of first-person POV images by having the authors wear an AR headset and play tennis. They then streamed video from a pair of AR glasses to a back-end server, analyzed the frames using a custom-trained deep learning model, and sent back the results for real-time overlaid visualization.
After a brainstorming session with an LV research team member, the team added visualization improvements to enhance the ball’s color contrast and add a crosshair in real-time.
Early evaluations have provided feedback that the prototype could help LV people enjoy ball-based sports but there’s plenty of further work to be done. A larger field-of-view (FOV) and audio cues would improve a player’s ability to track the ball. The weight and bulk of the headset, in addition to its expense are also factors the team expects to improve with time, as Lee noted in an interview on Oregon Public Broadcasting.
“Wearable AR devices such as the Microsoft HoloLens 2 hold immense potential in non-intrusively improving accessibility of everyday tasks. I view AR glasses as a technology that can enable continuous computer vision, which can empower BLV individuals to participate in day-to-day tasks, from sports to cooking. The Makeability Lab team and I hope to continue exploring this space to improve the accessibility of popular sports, such as tennis and basketball.”
Training a robot to feed people presents an array of challenges for researchers. Foods come in a nearly endless variety of shapes and states (liquid, solid, gelatinous), and each person has a unique set of needs and preferences. A team led by CREATE Ph.D. students Ethan K. Gordon and Amal Nanavati created a set of 11 actions a robotic arm can make to pick up nearly any food attainable by fork.
In tests with this set of actions, the robot picked up the foods more than 80% of the time, which is the user-specified benchmark for in-home use. The small set of actions allows the system to learn to pick up new foods during one meal. UW News talked with Gordon and Nanavati co-lead authors, both doctoral students in the Paul G. Allen School of Computer Science & Engineering, and with co-author Taylor Kessler Faulkner, a UW postdoctoral scholar in the Allen School, about the successes and challenges of robot-assisted feeding.
UW News talked with co-lead authors Gordon and Nanavati, both doctoral students members of CREATE and in the Paul G. Allen School of Computer Science & Engineering, and with co-author Taylor Kessler Faulkner, a UW postdoctoral scholar in the Allen School, about the successes and challenges of robot-assisted feeding for 1.8 million people in the U.S. (according to data from 2010) who can’t eat on their own.
The Personal Robotics Lab has been working on robot-assisted feeding for several years. What is the advance of this paper?
Ethan K. Gordon: I joined the Personal Robotics Lab at the end of 2018 when Siddhartha Srinivasa, a professor in the Allen School and senior author of our new study, and his team had created the first iteration of its robot system for assistive applications. The system was mounted on a wheelchair and could pick up a variety of fruits and vegetables on a plate. It was designed to identify how a person was sitting and take the food straight to their mouth. Since then, there have been quite a few iterations, mostly involving identifying a wide variety of food items on the plate. Now, the user with their assistive device can click on an image in the app, a grape for example, and the system can identify and pick that up.
Taylor Kessler Faulkner: Also, we’ve expanded the interface. Whatever accessibility systems people use to interact with their phones — mostly voice or mouth control navigation — they can use to control the app.
EKG: In this paper we just presented, we’ve gotten to the point where we can pick up nearly everything a fork can handle. So we can’t pick up soup, for example. But the robot can handle everything from mashed potatoes or noodles to a fruit salad to an actual vegetable salad, as well as pre-cut pizza or a sandwich or pieces of meat.
In previous work with the fruit salad, we looked at which trajectory the robot should take if it’s given an image of the food, but the set of trajectories we gave it was pretty limited. We were just changing the pitch of the fork. If you want to pick up a grape, for example, the fork’s tines need to go straight down, but for a banana they need to be at an angle, otherwise it will slide off. Then we worked on how much force we needed to apply for different foods.
In this new paper, we looked at how people pick up food, and used that data to generate a set of trajectories. We found a small number of motions that people actually use to eat and settled on 11 trajectories. So rather than just the simple up-down or coming in at an angle, it’s using scooping motions, or it’s wiggling inside of the food item to increase the strength of the contact. This small number still had the coverage to pick up a much greater array of foods.
We think the system is now at a point where it can be deployed for testing on people outside the research group. We can invite a user to the UW, and put the robot either on a wheelchair, if they have the mounting apparatus ready, or a tripod next to their wheelchair, and run through an entire meal.
For you as researchers, what are the vital challenges ahead to make this something people could use in their homes every day?
EKG: We’ve so far been talking about the problem of picking up the food, and there are more improvements that can be made here. Then there’s the whole other problem of getting the food to a person’s mouth, as well as how the person interfaces with the robot, and how much control the person has over this at least partially autonomous system.
TKF: Over the next couple of years, we’re hoping to personalize the robot to different people. Everyone eats a little bit differently. Amal did some really cool work on social dining that highlighted how people’s preferences are based on many factors, such as their social and physical situations. So we’re asking: How can we get input from the people who are eating? And how can the robot use that input to better adapt to the way each person wants to eat?
Amal Nanavati: There are several different dimensions that we might want to personalize. One is the user’s needs: How far the user can move their neck impacts how close the fork has to get to them. Some people have differential strength on different sides of their mouth, so the robot might need to feed them from a particular side of their mouth. There’s also an aspect of the physical environment. Users already have a bunch of assistive technologies, often mounted around their face if that’s the main part of their body that’s mobile. These technologies might be used to control their wheelchair, to interact with their phone, etc. Of course, we don’t want the robot interfering with any of those assistive technologies as it approaches their mouth.
There are also social considerations. For example, if I’m having a conversation with someone or at home watching TV, I don’t want the robot arm to come right in front of my face. Finally, there are personal preferences. For example, among users who can turn their head a little bit, some prefer to have the robot come from the front so they can keep an eye on the robot as it’s coming in. Others feel like that’s scary or distracting and prefer to have the bite come at them from the side.
A key research direction is understanding how we can create intuitive and transparent ways for the user to customize the robot to their own needs. We’re considering trade-offs between customization methods where the user is doing the customization, versus more robot-centered forms where, for example, the robot tries something and says, “Did you like it? Yes or no.” The goal is to understand how users feel about these different customization methods and which ones result in more customized trajectories.
What should the public understand about robot-assisted feeding, both in general and specifically the work your lab is doing?
EKG: It’s important to look not just at the technical challenges, but at the emotional scale of the problem. It’s not a small number of people who need help eating. There are various figures out there, but it’s over a million people in the U.S. Eating has to happen every single day. And to require someone else every single time you need to do that intimate and very necessary act can make people feel like a burden or self-conscious. So the whole community working towards assistive devices is really trying to help foster a sense of independence for people who have these kinds of physical mobility limitations.
AN: Even these seven-digit numbers don’t capture everyone. There are permanent disabilities, such as a spinal cord injury, but there are also temporary disabilities such as breaking your arm. All of us might face disability at some time as we age and we want to make sure that we have the tools necessary to ensure that we can all live dignified lives and independent lives. Also, unfortunately, even though technologies like this greatly improve people’s quality of life, it’s incredibly difficult to get them covered by U.S. insurance companies. I think more people knowing about the potential quality of life improvement will hopefully open up greater access.
Additional co-authors on the paper were Ramya Challa, who completed this research as an undergraduate student in the Allen School and is now at Oregon State University, and Bernie Zhu, a UW doctoral student in the Allen School. This research was partially funded by the National Science Foundation, the Office of Naval Research and Amazon.
Generative artificial intelligence tools like ChatGPT, an AI-powered language tool, and Midjourney, an AI-powered image generator, can potentially assist people with various disabilities. They could summarize content, compose messages, or describe images. Yet they also regularly spout inaccuracies and fail at basic reasoning, perpetuating ableist biases.
This year, seven CREATE researchers conducted a three-month autoethnographic study — drawing on their own experiences as people with and without disabilities — to test AI tools’ utility for accessibility. Though researchers found cases in which the tools were helpful, they also found significant problems with AI tools in most use cases, whether they were generating images, writing Slack messages, summarizing writing or trying to improve the accessibility of documents.
“When technology changes rapidly, there’s always a risk that disabled people get left behind,” said senior author Jennifer Mankoff, CREATE’s director and a professor in the Paul G. Allen School of Computer Science & Engineering. “I’m a really strong believer in the value of first-person accounts to help us understand things. Because our group had a large number of folks who could experience AI as disabled people and see what worked and what didn’t, we thought we had a unique opportunity to tell a story and learn about this.”
The group presented its research in seven vignettes, often amalgamating experiences into single accounts to preserve anonymity. For instance, in the first account, “Mia,” who has intermittent brain fog, deployed ChatPDF.com, which summarizes PDFs, to help with work. While the tool was occasionally accurate, it often gave “completely incorrect answers.” In one case, the tool was both inaccurate and ableist, changing a paper’s argument to sound like researchers should talk to caregivers instead of to chronically ill people. “Mia” was able to catch this, since the researcher knew the paper well, but Mankoff said such subtle errors are some of the “most insidious” problems with using AI, since they can easily go unnoticed.
Yet in the same vignette, “Mia” used chatbots to create and format references for a paper they were working on while experiencing brain fog. The AI models still made mistakes, but the technology proved useful in this case.
“When technology changes rapidly, there’s always a risk that disabled people get left behind.”
Jennifer Mankoff, CREATE Director, professor in the Allen School
Mankoff, who’s spoken publicly about having Lyme disease, contributed to this account. “Using AI for this task still required work, but it lessened the cognitive load. By switching from a ‘generation’ task to a ‘verification’ task, I was able to avoid some of the accessibility issues I was facing,” Mankoff said.
The results of the other tests researchers selected were equally mixed:
One author, who is autistic, found AI helped to write Slack messages at work without spending too much time troubling over the wording. Peers found the messages “robotic,” yet the tool still made the author feel more confident in these interactions.
Three authors tried using AI tools to increase the accessibility of content such as tables for a research paper or a slideshow for a class. The AI programs were able to state accessibility rules but couldn’t apply them consistently when creating content.
Image-generating AI tools helped an author with aphantasia (an inability to visualize) interpret imagery from books. Yet when they used the AI tool to create an illustration of “people with a variety of disabilities looking happy but not at a party,” the program could conjure only fraught images of people at a party that included ableist incongruities, such as a disembodied hand resting on a disembodied prosthetic leg.
“I was surprised at just how dramatically the results and outcomes varied, depending on the task,” said lead author Kate Glazko, a UW doctoral student in the Allen School. “”n some cases, such as creating a picture of people with disabilities looking happy, even with specific prompting — can you make it this way? — the results didn’t achieve what the authors wanted.”
The researchers note that more work is needed to develop solutions to problems the study revealed. One particularly complex problem involves developing new ways for people with disabilities to validate the products of AI tools, because in many cases when AI is used for accessibility, either the source document or the AI-generated result is inaccessible. This happened in the ableist summary ChatPDF gave “Mia” and when “Jay,” who is legally blind, used an AI tool to generate code for a data visualization. He could not verify the result himself, but a colleague said it “didn’t make any sense at all.” The frequency of AI-caused errors, Mankoff said, “makes research into accessible validation especially important.”
Mankoff also plans to research ways to document the kinds of ableism and inaccessibility present in AI-generated content, as well as investigate problems in other areas, such as AI-written code.
“Whenever software engineering practices change, there is a risk that apps and websites become less accessible if good defaults are not in place,” Glazko said. “For example, if AI-generated code were accessible by default, this could help developers to learn about and improve the accessibility of their apps and websites.”
Co-authors on this paper are Momona Yamagami, who completed this research as a UW postdoctoral scholar in the Allen School and is now at Rice University; Aashaka Desai, Kelly Avery Mack and Venkatesh Potluri, all UW doctoral students in the Allen School; and Xuhai Xu, who completed this work as a UW doctoral student in the Information School and is now at the Massachusetts Institute of Technology. This research was funded by Meta, Center for Research and Education on Accessible Technology and Experiences (CREATE), Google, an NIDILRR ARRT grant and the National Science Foundation.
The U.S. Department of Health and Human Services’ (HHS) Office for Civil Rights published a proposed update to the HHS regulations implementing Section 504 of the Rehabilitation Act of 1973, which prohibits disability discrimination by recipients of federal funding.
If you have any questions, reach out to CREATE at create-contact@uw.edu.
This is the first comprehensive update to the regulations since they were first put in place more than 40 years ago. The proposed rule includes new requirements prohibiting discrimination in the areas of:
Medical treatment
The use of value assessments
Web, mobile, and kiosk accessibility
Requirements for accessible medical equipment, so that persons with disabilities have an opportunity to participate in or benefit from health care programs and activities that is equal to the opportunity afforded others.
For 60 days starting on September 14, HHS will be seeking public comment on the proposed rule. Input from the disability and aging communities is essential!
Note that CREATE also provided a review guide and CREATE’s response in an accessible and tagged PDF document (53 pages) for a previous public comment invitation, specifically for the U.S. Department of Justice in the areas of digital accessibility.
The designers of the Virtual Traffic Stop app aim to ease tensions and prevent misunderstandings between drivers and law enforcement during traffic stops. For Hard-of-Hearing or Deaf drivers, the app can be used to communicate with law enforcement via chat during the video. Users can add family members and invite them to the chat for additional assistance.
A Gainesville Florida K-12 school has announced their endorsement of Virtual Traffic Stop and has encouraged parents and their children to sign up and start using the app. Currently, the app is being used by the University of Florida and Gainesville Florida police departments.
If your community is interested in using the app, contact Dr. Juan E. Gilbert, a former CREATE Advisory Board member and Chair of the Human-Centered Computing Department at the University of Florida, by calling 352-392-1527 or emailing juan@ufl.edu.
In September 2023, the Director of the National Institute on Minority Health and Health Disparities announced the designation of people with disabilities as a population with health disparities. The designation is one of several steps National Institutes of Health (NIH) is taking to address health disparities faced by people with disabilities and ensure their representation in NIH research.
Dr. Eliseo J. Pérez-Stable, in consultation with Dr. Robert Otto Valdez, the Director of the Agency for Healthcare Research and Quality cited careful consideration of the National Advisory Council on Minority Health and Health Disparities final report, input from the disability community, and a review of the science and evidence.
A proposal for new digital accessibility guidelines for entities receiving federal funds was released for review by the U.S. Department of Justice (DOJ) on August 4, 2023. Anyone affected by these guidelines has 60 days — through Tuesday, October 3, 2023 — to comment.
The DOJ is still trying to decide exactly what the rule should say, how quickly public entities should improve digital accessibility, and what exceptions to allow. For example, the current rule states that course content posted on a password-protected website (such as a learning management system (LMS) like Canvas) does not have to be made accessible until a student with a disability needs access to that content. If a student registers for the course, or transfers into it, then the course content has to be made fully accessible to all disabilities by the start of the term or within 5 days (if the term has already started). In addition, the course needs to stay accessible over time.
CREATE Director Jennifer Mankoff summarizes some of the most important aspects of the proposed rule in a Guide to Reviewing and Commenting that includes many of the the questions posed by the DOJ, with additional questions to consider from Mankoff. This guide is not meant to direct your comments, rather to facilitate and encourage your review. Whatever your viewpoint on the questions raised, the DOJ should hear from you.
CREATE researchers shone this spring at the 2023 Web4All 2023 conference that, in part, seeks to “make the internet more accessible to the more than one billion people who struggle to interact with digital content each day due to neurodivergence, disability or other impairments.” Two CREATE-funded open source projects won accolades.
Built on prior research to develop taxonomies of information sought by screen-reader users to interact with online data visualizations, the team’s research used these taxonomies to extend the functionality of VoxLens—an open-source multi-modal system that improves the accessibility of data visualizations—by supporting drilled-down information extraction. They assessed the performance of their VoxLens enhancements through task-based user studies with 10 screen-reader and 10 non-screen-reader users. Their enhancements “closed the gap” between the two groups by enabling screen-reader users to extract information with approximately the same accuracy as non-screen-reader users, reducing interaction time by 22% in the process.
Authors: Ather Sharif, Aneesha Ramesh, Qianqian Yu, Trung-Anh H. Nguyen, and Xuhai Xu
Ather Sharif’s work on another project, UnlockedMaps, was honored with the Accessibility Challenge Delegates’ Award. The paper details a web-based map that allows users to see in real time how accessible rail transit stations are in six North American cities, including Seattle, Toronto, New York and the Bay Area. UnlockedMaps shows whether stations are accessible and if they are currently experiencing elevator outages. Their work includes a public website that enables users to make informed decisions regarding their commute and an open source API that can be used by developers, disability advocates, and policy makers for a variety of purposes, including shedding light on the frequency of elevator outages and their repair times to identify the disparities between neighborhoods in a given city.
In June 2023, CREATE and HuskyADAPT co-hosted a showcase — and celebration — of outstanding UW research on accessible technology. The showcase featured poster presentations, live demonstrations by our faculty, students, and researchers and was altogether vibrant and exciting. Over 100 attendees viewed 25 projects, presentations, and posters.
Congratulations and appreciation to CREATE Engagement and Partnerships Manager Kathleen Quin Voss and HuskyAdapt Student Executive Chair Mia Hoffman for putting on an amazing research showcase!
Harniss’ professional background lies in special education and instructional technology, but his current focus revolves around knowledge translation, assistive technology, accessible information technology (IT), and disability law and policy.
In his role as CREATE Director for Education, Mark aims to foster collaboration and cooperation between UW “upper and lower campus,” particularly by forging connections between CREATE, the Disability Studies Program, the Institute on Human Development and Disability (IHDD), and the Department of Rehabilitation Medicine. Additionally, he intends to expand CREATE’s reach by establishing links with important external communities, ensuring that the innovations generated within CREATE are available to these communities. In turn, he envisions that these communities will provide valuable insights to CREATE researchers regarding their specific needs.
The CREATE community thanks three of our founding leaders for their energy and service in launching the center as we embark upon some transitions. “CREATE would not be where it is today without the vision, passion, and commitment that Jake, Richard, and Anat brought to their work leading the center,” says CREATE Director Jennifer Mankoff.
Co-Director Jacob O. Wobbrock: From vision, to launch, to sustainable leadership
It was back in June 2019 that Jacob O. Wobbrock, CREATE’s founding Co-Director, was on a panel discussion at Microsoft’s IdeaGen 2030 event, where he talked about ability-based design. Also on that panel was future CREATE Associate Director Kat Steele. After the event, the two talked with Microsoft Research colleagues, particularly Dr. Meredith Ringel Morris, about the possibility of founding an accessible technology research center at the University of Washington.
Wobbrock and Steele thought that a center could bring faculty together and make them more than the sum of their parts. Within a few months, Wobbrock returned to Microsoft with Jennifer Mankoff, Richard Ladner, and Anat Caspi to pitch Microsoft’s Chief Accessibility Officer, Jenny Lay-Flurrie, on the idea of supporting the new Center for Research and Education on Accessible Technology and Experiences (CREATE). With additional support from Microsoft President Brad Smith, and input from Morris, the center was launched by Smith and UW President Ana Marie Cauce at Microsoft’s Ability Summit in Spring 2020.
Wobbrock, along with Mankoff, served as CREATE’s inaugural co-directors until June 2023, when Wobbrock stepped down into an associate director role, with Mankoff leading CREATE as sole Director. “I’m a founder by nature,” Wobbrock said. “I helped start DUB, the MHCI+D degree, a startup called AnswerDash, and then CREATE. I really enjoy establishing new organizations and seeing them take flight. Now that CREATE is soaring, it’s time for more capable hands than mine to pilot the plane. Jennifer Mankoff is one of the best, most capable, energetic, and visionary leaders I know. She will take CREATE into its next chapter and I can’t wait to see what she does.” Wobbrock will still be very active with the center.
Professor Emeritus Richard Ladner, one of CREATE’s founders and our inaugural Education Director
We thank Professor Emeritus Richard Ladner for three years of leadership as one of our founders and CREATE’s inaugural Education Director. Ladner initiated the CREATE Student Minigrant Program that helps fund small grants up to $2,000 in support of student initiated research projects.
Ladner has shepherded 10 minigrants and worked directly with eight Teach Access Study Away students. Through his AccessComputing program, he helped fund several summer research internships for undergraduate students working with CREATE faculty. All CREATE faculty contribute to accessibility related education in their courses, where he provides encouragement.
Anat Caspi defined and elevated CREATE’s translation efforts, leveraging the center’s relationships with partners in industry, disability communities, and academia. Her leadership created sustainable models for translation and built on our prior successes. Collaborations with the TASKAR center, HuskyADAPT, and the UW Disability Studies Program have ensured diverse voices to inform innovation.
Director of Translation duties will be distributed across Mankoff, CREATE’s Community Engagement and Partnerships Manager Kathleen Quin Voss, and the Taskar Center for Accessible Technology, which Caspi directs.
Led by Human Centered Design and Engineering (HCDE) Ph.D. candidate Emma McDonnell and supported by CREATE, this work investigates how groups with both hearing and d/Deaf and hard of hearing (DHH) members could be better supported when using captions during videoconferences.
Researchers recruited four groups to participate in a series of codesign sessions, which de-centers researchers’ priorities and seeks to empower participants to lead the development of new design ideas. In the study, participants reflected on their experiences using captioning, sketched and discussed their ideas for technology that could help build accessible group norms, and then critiqued video prototypes researchers created of their ideas.
One major finding from this research is that participants’ relationships with each other shape what kinds of accessibility support the group would benefit from.
For example, one group that participated in our study were cousins who had been close since childhood. Now in their mid-twenties, they found they did not have to actively plan for accessibility; they had their ways of communicating and would stop and clarify if things broke down. On the other hand, a group of colleagues who work on technology for DHH people had many explicit norms they used to ensure communication accessibility. One participant, Blake, noted,“I was pretty emotional after the first meeting because it was just so inclusive.” These different approaches demonstrate that there is no one-size-fits-all approach to communication accessibility – people work together as a group to develop an approach that works for them.
This paper also contributes new priorities for the design of videoconferencing software. Participants focused on designing add-ons to videoconferencing systems that would better support their group in communicating accessibly. Their designs fell into four categories:
Speaker Identity and Overlap: Having video conferencing tools identify speakers and warn groups when multiple people speak at once, since overlapping speech can’t be captioned accurately. Participants found this to be critical, and often missing, information.
Support for Behavioral Feedback: Building in ways for people to subtly notify conversation partners if they need to adjust their behavior. Participants desired tools to flag when people need to adjust their cameras, critical caption errors, and if speech rate gets too high. They considered, but decided against, a general purpose conversation breakdown warning.
Videoconferencing Infrastructure for Accessibility: Adding more features and configurable settings around conversational accessibility to videoconferencing platforms. Participants desired basic controls, such as color and font size, as well as the ability to preset and share group accessibility norms and customize behavior feedback tools.
Sound Information: Providing more information about the sound happening during a conversation. Participants were excited about building sound recognition into captioning tools, and considered conveying speech volume via font weight, but decided it would be overwhelming and ambiguous.
This research also has implications for broader captioning and videoconferencing design. While often captioning tools are designed for individual d/Deaf and hard of hearing people, researchers argue that we should design for the entire group having a conversation. This shift in focus revealed many ways that, on top of transcribing a conversation, technology could help groups communicate in ways that can be more effectively captioned. Many of these tools are easy to build with current technology, such as being able to click on a confusing caption to request clarification. The research team hopes that their work can illuminate the need to pay attention to groups’ social context when studying captioning and can provide videoconferencing platform designers a design approach to better support groups with mixed hearing abilities.
McDonnell is advised by CREATE Associate Directors Leah Findlater, HCDE, and Jon Froehlich, Paul G. Allen School of Computer Science & Engineering.
