Student exploring how AI can assist people with vision loss
Kelly Raines, a student in the computer science accelerated master’s degree program in the School of Computing and Augmented Intelligence, part of the Ira A. Fulton Schools of Engineering at Arizona State University, poses in a lab wearing a pair of smart glasses. She studies how artificial intelligence can assist people with visual disabilities. Photo by Kelly deVos/ASU
Partial vision loss can make life challenging for more than 6 million Americans. People with visual disabilities that can’t be remedied with glasses or contacts can sometimes struggle to safely navigate the world, with some having trouble reading street signs or dealing with changes in terrain while walking.
A student in the School of Computing and Augmented Intelligence, part of the Ira A. Fulton Schools of Engineering at Arizona State University, studies ways to use artificial intelligence, or AI, to help people with visual disabilities more fully experience the world around them.
Kelly Raines is working on her master’s degree in computer science through the Fulton Schools Accelerated Master’s degree program. After being awarded her undergraduate degree in computer science with an emphasis in software engineering in fall 2024, Raines joined the program, which allows engineering students to earn a master’s degree with as little as one additional year of study.
As an undergraduate student, she connected with opportunities in the Laboratory for Learning Evaluation of autoNomous Systems, or LENS Lab, a research group led by Ransalu Senanayake, a Fulton Schools assistant professor of computer science and engineering. Under Senanayake’s supervision, Raines developed AI that can work with smart glasses, including the Ray-Ban Meta AI Glasses, to assist those with visual disabilities by allowing them to ask questions about their environment and receive assistance in making informed decisions.
In January, Raines was given special recognition for the work from the Computing Research Association, or CRA, where she received an honorable mention in the Outstanding Undergraduate Researcher Awards. The award program is designed to acknowledge and foster the development of talented North American computer science students.
Seeing real solutions in artificial intelligence
The LENS Lab project seeks to leverage the capabilities that smart glasses already have and combine them with new features specially designed to assist those with visual disabilities. Smart glasses can already capture photos and video, recognize common objects in them and combine digital information with real-world images.
In the early phases of the project and for prior work, Raines conducted interviews with members of the visually disabled and blind communities to better understand their needs and experiences. The team recognized that safety and reliability were top concerns when designing.
Raines and the team then created an AI-powered assistant that acts as a visual guide. As the smart glasses collect images, the wearer can speak questions aloud, asking for more details. A person might ask the AI assistant how many steps of a staircase are ahead, or to identify an object off in the distance or to read the text of a street sign.
Raines explains that the team’s goal was to combine different pieces of technology that are already available to create new solutions.
“My research explored the current capabilities of state-of-the-art AI models,” she says. “We wanted to use the best models to accurately describe the environment, understand spatial reasoning and assist with other important tasks in navigation.”
Senanayake says he is proud of Raines’ work and believes that research can play an important part in student development.
“Kelly is a great student and an engaged researcher who worked hard on this project,” Senanayake says. “Her award from the CRA demonstrates how ASU students are combining their creativity, enthusiasm and technical skills to create new forms of game-changing AI technology that empower differently abled individuals, offering greater independence and accessibility in everyday life.”
Raines agrees and says that research opportunities help students find a sense of purpose.
“I did not have family in the tech field, nor did I really grow up interacting with programming until college. So I felt like everything was so unfamiliar and new. It was sometimes almost daunting to try to start,” she says. “But research is one of the most beneficial pathways to discover topics that you enjoy.”
Techie of tomorrow eyes career creating positive change
While working in the LENS Lab, Raines also co-authored a paper published at a top research conference and landed an internship at Viasat, assisting on AI projects for the satellite internet company.
Given the CRA’s role in computing research, Raines’ latest honor is especially significant. The group is a consortium of more than 250 organizations active in computing research and dedicated specifically to research advancement.
Nadya Bliss is the executive director of the ASU Global Security Initiative and also serves as chair of the CRA Computing Community Consortium. In both roles, fostering student participation in computer science research is key.
She says this recognition is an encouraging validation of the efforts of students and faculty members in the School of Computing and Augmented Intelligence.
“These awards honor undergraduate students who show real promise and passion for computing research,” Bliss says. “This recognition demonstrates that ASU faculty members are creating challenging learning experiences and students are rising to the occasion to create impactful work.”
For her part, Raines plans to continue to find more ways to use AI to make a difference.
“I hope to utilize my master’s degree in all sorts of media, interactive environments or creative works that can create a positive impact on the world,” she says.
More Science and technology
Ancient sea creatures offer fresh insights into cancer
Sponges are among the oldest animals on Earth, dating back at least 600 million years. Comprising thousands of species, some with lifespans of up to 10,000 years, they are a biological enigma.…
When is a tomato more than a tomato? Crow guides class to a wider view of technology
How is a tomato a type of technology?Arizona State University President Michael Crow stood in front of a classroom full of students, holding up a tomato.“This object does not exist in nature,” he…
Newly identified viruses found in dolphins
In a new study, researchers from Arizona State University together with national and international collaborators have identified anelloviruses in dolphins for the first time. Anelloviruses are a…