Meta-AI researchers claim they’ve built an artificial visual cortex. This should help robots to interact with their environment using camera images.

In the human brain, the visual cortex or cortex is responsible for processing images. In order to build robots that can perform tasks in the real world, an artificial alternative is needed. Meta researchers now claim to have built such an artificial visual cortex. The result is called VC-1 and is based on the Edgo4D dataset.

VC-1 allows an AI to process visual information in a meaningful way. The Ego4D dataset on which the model was trained therefore contains thousands of hours of video from handheld cameras. The images were filmed by participants on behalf of researchers while completing daily tasks. Think of cooking or cleaning.

First see, then understand

So VC-1 should help robots to understand what they see. Getting started with that is the next step. AI needs to be trained to effectively perform tasks based on visual information. Meta announces a breakthrough in this area as well. Researchers, together with the Georgia Institute of Technology, are said to have developed a system in which AI is trained in simulations in order to then transfer the learned experiences to a robot that has to find its way in the real world.

Meta tested the system with Boston Dynamics’ robotic dog Spot. In the simulation, Spot was digitally trained to walk through virtual houses and find and move certain objects there. Over time, Spot learned what “houses” looked like in general. Armed with this knowledge, Spot could perform the same task in a real apartment.

According to Meta, VC-1 will be open source. The company will now focus on integrating VC-1 with simulated learning.