“What’s over there?” “How do I solve this math problem?”
If you try asking a voice assistant (VA) like Siri or Alexa such questions you won’t get much information. While VAs are transforming human-computer interaction, they can’t see what you’re looking at or where you’re pointing. CREATE Ph.D. student Jaewook Lee has led an evaluation of GazePointAR, a fully-functional, context-aware VA for wearable augmented reality (AR) that uses eye gaze, pointing gestures, and conversation history to make sense of spoken questions.
An example interaction with GazePointAR. The user’s query “What is this?” is automatically resolved by using real-time gaze tracking, pointing gesture recognition, and computer vision to replace “this” with “packaged item with text that says orion pocachip original,” which is then sent to a large language model for processing and the response read by a text-to-speech engine.
Lee, along with advisor Jon E. Froehlich and fellow researchers, evaluated GazePointAR by comparing it to two commercial systems. The team also studied GazePointAR’s pronoun handling across three assigned tasks and its responses to participants’ own questions. In short, participants appreciated the naturalness and human-like nature of pronoun-driven queries, although sometimes pronoun use was counter-intuitive.
AccessMap is an app and a collection of data sets that let pedestrians tailor routes for their accessibility needs and preferences. Coverage has been spreading across Washington state and now includes a new data set, called OS-CONNECT, for sidewalks and other paths statewide, from Forks on the Olympic Peninsula to Clarkston in the southeast.
“Not only are we including all sidewalks in Washington, which is huge, but we are engaging communities and planners in a massive effort to support data production and the maintenance of this resource long term, to make it sustainable and translatable to other institutions. This way states across the U.S. could start using it.”
Anat Caspi, Director, Taskar Center and a CREATE associate director
The Washington State Legislature (in House Bill 1125) assigned TCAT to build the OS-CONNECT data set, which the team completed well ahead of its projected 2027 goal. The team will now perform deep quality checks, work with the different communities to analyze and interpret what the data means to them, and engage citizens in actions that promote public participation in data and active transportation.
Interactive map of Washington state. Mapped cities and towns are outlined in grey.
The same map, zoomed in to the Olympic Peninsula, east to Bellevue. The legend is open to show meaning of different route colors.
The Taskar Center launched OS-Connect at its annual OpenThePaths conference in March. The conference brings together community members, advocates, planners, researchers and policymakers dedicated to expanding and sustaining pedestrian-friendly infrastructure.
“No state has before used machine learning and human vetting to collect, in a consistent, standardized way, all of the pedestrian infrastructure in that state,” said Caspi, a research principal in the Paul G. Allen School of Computer Science & Engineering, where TCAT is housed. “OS-CONNECT helps us answer the original question the state asked: ‘Who has access to frequent transit?’ And now we can answer many other questions, such as: ‘What type of access do people with diverse needs have to important services like grocery stores, schools and health care?’”
The state compiled OS-CONNECT using TCAT’s OpenSidewalks model, which combines machine learning with human vetting to catalogue pedestrian infrastructure. For instance, using the data set through the AccessMap app, a person using a wheelchair can plan a route only on streets that have sidewalks, don’t have an incline of greater than 5% and have curb ramps for any intersections.
The data can help local governments identify where sidewalks are in poor condition or missing. OS-CONNECT supports Walkshed, an accessibility app for urban planners, and projects such as Complete Streets, a model for equitable infrastructure design, and Vision Zero, a Seattle project to end traffic deaths and serious injuries by 2030.
CREATE students are encouraged to join the HuskyADAPT leadership team through a variety of student chair positions. Applications are being accepted through Friday, April 18.
Being on the leadership team involves approximately 5 hours per week of commitment, though this may vary based on your role and upcoming events. The HuskyADAPT chairs are the student leaders who plan and facilitate all HuskyADAPT events, as demonstrated at the Spring Community Meeting.
HuskyADAPT student executive chairs
Coordinates Student Exec Board meetings, serves as the main HuskyADAPT contact, communicates with faculty advisors, organizes finances and spearheads funding applications, and mentors/supports other officers. Useful skills/experiences include leadership (especially previous leadership with HuskyADAPT), communication, organization, and event management.
Manages HuskyADAPT budgets, approving purchases, applying for new funding opportunities, organizing our yearly giving campaign, and compiling our annual report.
Lead on-campus and off-campus toy adaptation efforts. Essential skills include the ability to plan and facilitate events (2 events per month, 20 to 50 people per event), expertise in toy adaptation, strong organizational skills, and patience. Helpful experiences include regular attendance at toy adaptation events. Toy adaptation chairs work closely together.
Lead our Yellow Toy Club and help to foster a toy-adaptation community! Plan weekly Yellow Toy Club meetings, manage the club space, and lead twice a quarter Yellow Toy Fix It Events. Essential skills include expertise in toy adaptation and organization skills. Helpful experiences include regular attendance at toy adaptation events, Lead Toy Adapter certification, and has participated in Yellow Toy Club.
Lead toy donation and distribution (including leading collaborations with toy library organizations such as the PNW Adapted Toy Library and a growing collaboration with King County Library System. Essential skills include expertise in toy adaptation and strong communication and organizational skills. Helpful experiences include regular attendance at toy adaptation events.
Integrate the UW GoBabyGo Leadership team with HuskyADAPT. This may include coordinating volunteers and necessary logistics for GoBabyGo workshops, assisting with marketing development for the GoBabyGo program and workshops, supporting the GoBabyGo design team, and general involvement with the GoBabyGo Leadership team. Helpful experiences/skills include knowledge of the national GoBabyGo organization, ability to plan and lead events, and passion for inclusion and importance of early mobility for children with disabilities. Previous toy or car adaptation experience or electrical/wiring experience a plus! This position will also require additional planning meetings with the Co-directors of UW GoBabyGo from Rehab Medicine, Shawn Rundell and Heather Feldner.
Lead student design projects that tackle accessibility challenges in our community. This includes determining suitable design projects, creating design teams, and mentoring and teaching technical skills to design teams. Essential skills include design knowledge, human-centered design experience, and organizational skills. Recommended prior experience in a human-centered design or HuskyADAPT Design Team.
Lead communications! Manage and update the HuskyADAPT website and social media pages (Twitter, Instagram), create flyers about events occurring throughout the year, and send the bi-weekly emails. Helpful skills include graphic design, social media skills, organization and communication skills.
Lead outreach through partnerships with K-12 students, industry partners, and clinical partners. Note that this includes all toy adaptation and design events held offcampus as well as on-campus K-12 outreach such as tours and Discovery Days. Help connect us with other organizations on campus, such as CREATE and the D-Center. Attend monthly Engineering Student Council meetings and CREATE events. Lead special events, like our twice-a-year Design-a-thon and Design Showcases. Essential skills include ability to plan and facilitate events, expertise in toy adaptation, and strong organizational and communication skills. Helpful experiences include enjoying teaching and working with children.
Don't miss this opportunity to get more involved in CREATE and help HuskyADAPT continue its impactful work!
HuskyADAPT is a student organization, supported by CREATE, that collaborates with individuals with disabilities and community partners on design projects. They provide adapted toys and devices for free to the local community, create low-tech adaptive technologies such as adapted books and low-cost switches, and offer education on accessible design on campus and in the community.
Skill Development: Gain hands-on experience in accessible design, project management, mentoring, and community outreach.
Networking: Connect with professionals, community partners, and fellow students passionate about accessibility.
Leadership Experience: Enhance your leadership skills by taking on roles such as toy donation chair, student executive chair, or outreach chair, which are well-suited to graduate student schedules and responsibilities.
As a current CREATE Ph.D. student and long-time HuskyADAPT leader, I appreciate the opportunity to be involved in the local disability community AND teach others about accessibility. As someone who does research with a lot of local families and young children, I always share HuskyADAPT as a resource to them as well.
Papers, presentations, and workshops from CREATE researchers at CHI 2025, the ACM CHI conference on Human Factors in Computing Systems. We appreciate your patience as we continue to update this page.
The conference takes place April 26 - May 1 in Yokohama Japan.
James Fogarty, a CREATE associate director, elected to the ACM SIGCHI Academy Class of 2025.
Cecilia Aragon, a new CREATE faculty member, received a SIGCHI Special Recognition for "establishing human-centered data science as a new field bridging HCI and data science, demonstrating its impact through applications from astrophysics to energy systems."
UW colleagues Kate Starbird (elected to the SIGCHI Academy); Nadya Peek (Special Recognition); Alexis Hiniker (Societal Impact Award).
Papers
A Stakeholder Value Framework for Augmentative and Alternative Communication
"A Tool for Freedom": Co-Designing Mobility Aid Improvements Using Personal Fabrication and Physical Interface Modules With Primarily Young Adults
Jerry Cao (CREATE Ph.D. student), Krish Jain, Julie Zhang, Yuecheng Peng, Shwetak N Patel, Jennifer Mankoff (CREATE Director).
Accessibility for Whom? Perceptions of Mobility Barriers Across Disability Groups and Implications for Designing Personalized Maps
Chu Li (CREATE Ph.D. student), Rock Yuren Pang (CREATE Ph.D. student), Delphine Labbé, Yochai Eisenberg, Maryam Hosseini, Jon E. Froehlich (CREATE associate director).
Autoethnographic Insights from Neurodivergent GAI "Power Users"
Kate S. Glazko (CREATE Ph.D. student), JunHyeok Cha, Aaleyah Lewis (CREATE Ph.D. student), Ben Kosa, Brianna L. Wimer, Andrew Zheng, Roy Zheng, Jennifer Mankoff (CREATE Director).
Cultivating Computational Thinking and Social Play Among Neurodiverse Preschoolers in Inclusive Classrooms
Maitraye Das (former CREATE postdoc), Megan Tran, Amanda Chih-han Ong, Julie A Kientz (CREATE faculty), Heather Feldner (CREATE associate director).
Deploying and Examining Beacon for At-Home Patient Self-Monitoring With Critical Flicker Frequency
Richard Li (CREATE Ph.D. student), Philip Vutien; Sabrina Omer, Michael Yacoub, George Ioannou, Ravi Karkar, Sean A Munson, James Fogarty (CREATE associate director).
Exploring AI-Based Support in Speech-Language Pathology for Culturally and Linguistically Diverse Children
Aaleyah Lewis (CREATE Ph.D. student), Aayushi Dangol; Hyewon Suh; Abbie Olszewski; James Fogarty (CREATE associate director); Julie A. Kientz (CREATE faculty).
Exploring Reduced Feature Sets for American Sign Language Dictionaries
Ben Kosa, Aashaka Desai (CREATE Ph.D. student), Alex X Lu, Richard E Ladner (CREATE Director for Education Emeritus), Danielle Bragg.
"I want to think like an SLP": A Design Exploration of AI-Supported Home Practice in Speech Therapy
Aayushi Dangol, Aaleyah Lewis (CREATE Ph.D. student), Hyewon Suh, Xuesi Hong, Hedda Meadan, James Fogarty (CREATE associate director), Julie A. Kientz (CREATE faculty).
Inaccessible and Deceptive: Examining Experiences of Deceptive Design with People Who Use Visual Accessibility Technology
Wordplay: Accessible, Multilingual Interactive Typography. ACM SIGCHI Conference on Human Factors in Computing Systems
Amy J. Ko (CREATE faculty), Carlos Aldana Lira, Isabel Amaya.
Presentations
To be announced
Workshops
To be announced
Student Games Competition
Accepted entries in the Student Games Competition (SGC) are the top 10 finalists from two tracks: Transformative and Transgressive Play and Innovative Interfaces. Entries will be further judged at the conference, and a winner chosen for each track.
OURCADE: A Game to Solve Real-World Game Accessibility Puzzles Jesse Martinez | Transformative and Transgressive Play track
The UW’s Population Health Initiative announced two fellowships that may be of interest to CREATE graduate and undergraduate students.
Accessibility of King County parks
Graduate and undergraduate students from all UW schools and colleges are encouraged to apply for the Population Health Applied Research Fellowship on the accessibility of King County Parks. The Summer 2025 fellowship team will collaborate with the King County Demographer and King County Parks to assess park accessibility.
This paid fellowship program offers training in data analysis techniques as well as in research and presentation skills, while they develop a work product for an external partner. Students will combine quantitative methods with field research to provide insights into physical access networks around parks, helping prioritize improvements to ensure all residents can enjoy these green spaces.
Register to attend the March 3, 12–1 p.m. info session, in person on the Seattle campus or virtually.
Four fellows will be selected from a variety of disciplines to investigate projects focused on finding innovative ways to maintain the balance between financial sustainability and social impact. Of particular interest:
Virtual Study Assistant for Potential Research Participants, a bilingual virtual study assistant that seeks to support recruitment and screening in research studies, leveraging AI to improve accessibility and reduce resource needs while ensuring careful consideration of potential biases in machine translations.
The Population Health Initiative is partnering with the UW’s Buerk Center for Entrepreneurship and CoMotion to offer its summer Social Entrepreneurship Fellows Program.
