Real-time descriptions of surroundings for people who are blind [Video]

The quick and clear mental image of the real world helps people who are blind or have low vision focus on other tasks, or just enjoy the things around them

A world of color and texture could soon become more accessible to people who are blind or have low vision through new software that narrates what a camera records.

The tool, called WorldScribe, was designed by University of Michigan researchers and will be presented at the ACM Symposium on User Interface Software and Technology in Pittsburgh next week.

The tool uses generative AI (GenAI) language models to interpret the camera images and produce text and audio descriptions in real time to help users become aware of their surroundings more quickly. It can adjust the level of detail based on the user’s commands or the length of time that an object is in the camera frame, and the volume automatically adapts to noisy …