Scientists have developed a robotic sensor that can recognize Braille alphabet twice as fast as most humans. For this, it uses not only unusual tactile sensors, but also a highly efficient neural network.

Robotics is actively developing, and the tasks it faces require the creation of algorithms with some characteristics of the human body. For example, to automate delivery, it is necessary to improve the ability of robots to navigate in space. And in order to master the resources of the oceans, it is necessary to teach robots tactile perception, that is, to identify objects by touch. Scientists have already developed a “magnetic skin” that allows you to do this: with its help, the algorithm can at least distinguish between spiritual and inanimate objects.

Thanks to the sensitivity of the fingers, a person receives a huge amount of information about the surrounding world. At the same time, the softness of the skin allows you to regulate the interaction with the environment: for example, the force with which people squeeze objects. It is difficult to create a technology that has both of these features at the same time. But scientists from the University of Cambridge (Great Britain) managed to do this at a high level.

In an article published by the magazine IEEE Robotics and Automation LettersResearchers have described a robotic sensor that recognizes Braille more accurately and faster than analogs. Its predecessors read one letter at a time and left the surface after each symbol; people perceived Braille text differently. The new technology works with the font almost like a human, but does it twice as fast. A neural network model was used for braille recognition YOLO v8 Relatively undemanding to computing resources.

According to scientists, the robot processes information from the camera and touch sensors simultaneously, but when it moves, the image becomes blurry and it becomes difficult to recognize the letters. To avoid this problem, the researchers trained the model on artificially blurred text written in Braille. Thanks to computer vision technology, the algorithm was able to detect and classify each character.

The resulting robot was tested with rapidly moving lines of Braille text. The sensor reads 315 words per minute with 87% accuracy. It’s twice as fast as most people who understand the font and has almost the same number of errors. Scientists noted that such a result could be considered surprising when artificial blur was used.

Initially, the technology wasn’t intended to be useful: Reading Braille is an effective way to test the sensitivity of the sensor. Such developments are useful, for example, for prosthetics with artificial skin, as well as for applications that determine surface texture. However, the last algorithm can become an assistant for people with visual impairment: scientists managed to achieve a balance of speed and accuracy, the indicators of which are close to human.