Researchers at Delft University of Technology have created a drone that can fly autonomously using neuromorphic image processing and control inspired by the workings of animal brains. Animal brains typically process data and consume energy more efficiently than deep neural networks running on GPUs. Therefore, neuromorphic processors are ideal for small drones because they eliminate the need for large hardware and large batteries.

The results are extraordinary: In flight, the drone’s deep neural network processes data 64 times faster and consumes three times less energy than running on the GPU. Further development of this technology could allow drones to be as small, agile and intelligent as flying insects or birds. The findings were recently published Science Robotics.

Learning from animal brains: advanced neural networks

Artificial intelligence has great potential to provide autonomous robots with the intelligence needed for real-world applications. However, modern AI relies on deep neural networks, which require significant computing power. Processors designed to run deep neural networks (GPUs, GPUs) consume significant amounts of energy. This is especially a problem for small robots such as flying drones because they may have very limited resources in terms of sensing and computation.

Animal brains process information in a very different way than neural networks running on GPUs. Biological neurons process information asynchronously and communicate mostly using electrical impulses. adhesions . Because sending such bursts requires energy, the brain minimizes bursts, causing processing to become sparse.

Scientists and technology companies are inspired by these features of the animal brain and develop new ones. neuromorphic processors These new processors allow running neural networks and promise to be much faster and more energy efficient.

One of the paper’s authors, Ph.D. “Computations performed using spike neural networks are much simpler than standard deep neural networks,” says Jesse Hagenaars. — Digital spiking neurons only need to add integers, while standard neurons need to multiply and add floating-point numbers. This makes neural networks faster and more energy efficient. To understand why, consider that it is much easier for humans to calculate 5 + 8 than it is to calculate 6.25 × 3.45 + 4.05 × 3.45.”

This energy efficiency is further increased when neuromorphic processors are used in conjunction with neuromorphic sensors such as neuromorphic cameras. These types of cameras do not create images in a fixed time interval. Instead, each pixel only sends out a signal when it gets brighter or darker. The advantages of such cameras are that they detect motion much faster, are more energy efficient, and perform well in both dark and bright environments. Additionally, signals from neuromorphic cameras can be sent directly to impulse neural networks running on neuromorphic processors. Together, they could form a massive vehicle for autonomous robots, especially small, agile robots like flying drones.

First neuromorphic vision and control of a flying drone

In the article published in Science RoboticsResearchers at Delft University of Technology in the Netherlands demonstrate for the first time a drone that uses neuromorphic vision and control for autonomous flight. Specifically, they developed a pulsed neural network that processes signals from a neuromorphic camera and issues control commands that determine the drone’s stance and thrust. They embedded this network into a neuromorphic processor on the drone called the Intel Loihi Neuromorphic Research Chip. Thanks to the network, the drone can detect and control its own movement in all directions.

“We faced many problems,” says Federico Paredes-Valles, one of the researchers working on the study, “but the most difficult was to imagine.” like we can train a bursty neural network so that learning is fast enough And learned networking will work well in real business. As a result, we developed a network consisting of two modules. The first module learns to visually detect motion based on signals from a moving neuromorphic camera. It does this entirely on its own, in a self-monitoring manner, relying solely on data from the camera. This is similar to how animals learn to perceive the world on their own. The second module learns to display the calculated movement to control commands in the simulator. This training was based on an artificial evolution simulation in which networks that control the drone better have a higher chance of producing offspring. Over many generations of artificial evolution, impulse neural networks got better and better at controlling it, eventually managing to fly at different speeds in any direction. We taught both modules and developed a way to combine them. “We were very pleased to see that the combined network worked immediately in real operation.”

Thanks to neuromorphic vision and control, the drone can fly at different speeds in different lighting conditions, from dark to light. It can even fly with flashing lights that cause the neuromorphic camera pixels to send numerous non-motion-related signals to the network.

Improved energy efficiency and speed thanks to neuromorphic artificial intelligence

“Importantly, our measurements confirm the potential of neuromorphic AI. The network launches on average 274 to 1600 times per second. If we run the same network on a small embedded GPU, it runs an average of only 25 times per second, a difference of about 10-64 times! Additionally, Intel Loihi neuromorphic research Its chip consumes 1,007 watts when online, of which 1 watt is idle power consumed by the processor when the chip is turned on. In comparison, the integrated GPU consumes 3 watts when running on the same network, of which 1 watt is standby power. “The neuromorphic approach enables AI to work faster and more efficiently, allowing it to be applied to much smaller autonomous robots,” says Stein Stroobants, Ph.D. in the field of neuromorphic drones.

Future applications of neuromorphic artificial intelligence for small robots

“Neuromorphic artificial intelligence will allow all autonomous robots to become smarter,” says biological drone professor Guido de Kroon. “But this is an absolute boost for small autonomous robots. At the Department of Aerospace Engineering at Delft University of Technology, he studies crops in greenhouses.” We are working on small autonomous drones that can be used for a variety of tasks, from monitoring to tracking inventory in warehouses. The advantages of tiny drones are that they are very safe and can navigate in tight environments such as tomato beds. They can also be very cheap, so they can be deployed in swarms. This is as we demonstrated in reconnaissance settings and locating the source of the gas. , useful for faster coverage of the area.”

“The current study is a major step in this direction. But implementation of these programs will depend on further reducing neuromorphic hardware and expanding capabilities for more complex tasks such as navigation.”