In an interesting take on neuroplasticity, neuroscientists have shown that blind people recognize basic faces using the same regions of the brain as sighted people, even when the shapes of the faces are transmitted as sound rather than through the visual cortex.

The ability to recognize faces is deeply rooted in humans as well as in some of our distant, socially oriented primate relatives. Indeed, there appear to be areas in the brain that light up specifically when we see faces (specifically, a spot in the back of the brain, in the inferior temporal cortex, called the fusiform face area, or FFA).

Interestingly, a 2009 study found that the FFA was activated even when people saw things that even vaguely resembled faces; This means it plays a role in the phenomenon of pareidolia, when we see faces in inanimate objects. This same area also begins to activate when people begin to develop expertise in a particular area; apparently it helps car enthusiasts distinguish different models by sight, or helps chess experts recognize a familiar configuration on the board.

It is noteworthy that FFA also responds to people who are congenitally blind; A 2020 MIT study placed blind people in an fMRI scanner and had them feel different 3D-printed shapes, including faces, hands, chairs, and mazes, and found that touching these tiny faces activated FFA in a similar way.

So FFA seems to sort of not care which sensory system feeds facial information, and a new study by a team of neuroscientists at Georgetown University Medical Center adds evidence to this hypothesis.

The team recruited six blind and 10 sighted people and began training them with a “sensory substitution device.” This includes a head-mounted video camera, blindfolded goggles, a headset, and a data-processing computer that will take input from the video camera, convert it into sound, divide the field of view into a 64-pixel grid, and display each image. the pixel’s own tone of voice.

These heights were also represented on the stereo soundstage; Therefore, according to the research paper, “if the image is just a dot located in the upper right corner of the camera’s field of view, the corresponding sound will be high frequency. and is transmitted primarily through the right earphone. If the dot is in the upper center of the field of view, the sound will be high-pitched but will come from the right and left headphones at the same volume. If the image is a line in the lower left corner, the corresponding sound will be a mixture of low frequencies coming mainly from the left earphone.”

The subjects completed 10 one-hour training sessions with these devices and learned to “see” with their ears while moving their heads. The cards have simple shapes; horizontal and vertical lines, houses of different shapes, geometric shapes and basic happy and sad faces in emoji style. It was a very difficult learning curve, but eventually all subjects recognized the simple shapes with over 85% accuracy.

In the fMRI shape recognition test, both sighted and blind subjects showed FFA activation when a basic face shape was presented. Some blind participants were also able to accurately detect whether a face was happy or sad, as you can hear in the study’s 45-second audio recording. This will also give you an idea about the sounds of the device.

“Our results from blind people suggest that the development of the fusiform facial region does not depend on the experience of visually seeing faces, but rather on the influence of the geometry of the facial configuration, which can be conveyed by other sensory modalities.” Josef Rauschecker, Ph.D., professor of neuroscience and senior author of the study, said in a press release.

The team also found that sighted subjects experienced activation mainly in the right fusiform facial region, while blind subjects experienced activation in the left FFA.

“We think the difference between blind and blind people may be related to how the left and right sides of the fusiform area are processed as connected patterns or separate parts, which could be an important clue to help us develop our sensor replacement device,” says Rauschecker.

The team wants to continue the experiments by developing a higher-resolution sensory modification device that will allow highly trained subjects to recognize real human faces.

Note that such video-to-sound devices are unlikely to be helpful in a practical sense; This is partly because they require a lot of training and partly because blind people already rely so much on their hearing and don’t want the extra beeps. mistakes that distort their perception of the world.

Not to mention that with the advent of multimodal deep learning AI, systems already exist that allow GPT-style language models to look at images or videos and explain what’s happening with your preferred level of detail. This type of natural language storytelling can be much easier to implement, use, and adapt to human needs than direct video-to-audio conversion.

This is all pretty fascinating stuff, and shows how deeply embedded the ancient form of two eyes and one mouth is in our hardware, and the importance of these forms to us as social animals. Source