AI recognizes and learns to predict behavioral patterns based on video

Researchers at Carnegie Mellon University, University Hospital Bonn, and the University of Bonn have created an open-source platform known as A-SOiD that can learn and predict user-defined behaviors from video alone. The results of the research are now published in the journal Nature Methods.

“This technique is great for studying the classification of different types of behavior in animals and humans,” said Eric Ittree, the Eberly Family Associate Professor of Biological Sciences at Carnegie Mellon. “It will work not just on behavior, but on the behavior of anything if there are identifiable patterns: stock markets, earthquakes, proteomics. It’s a powerful pattern recognition machine.”

Unlike many artificial intelligence (AI) programs, A-SOiD is not a black box. Instead, the researchers allowed the program to relearn what it did wrong. They first trained the program on a portion of the dataset and focused on the program’s weak beliefs. If the program is uncertain, the algorithm will build confidence in the training data.

Because A-SOiD was taught to focus on algorithmic uncertainty rather than treating all data equally, he recently received a Ph.D. Alex Hsu is. The Carnegie Mellon graduate said it avoids typical biases found in other AI models.

AI tool matches every class in the dataset

“This is a different way of presenting data,” Hsu said. “Often people use the entire data set for the behavior they are looking for. They rarely realize that the data may be unbalanced, meaning that there may be well-represented behaviors in their set and poorly represented behaviors in their set. This bias can then spread from the prediction process to the experimental results. Our algorithm “It deals with balancing the data by learning only the weaker ones. Our method more fairly represents each class in the dataset.”

Since A-SOiD is supervised learning, it can provide highly accurate results. Given a set of data, it can tell the difference between a normal human tremor and a Parkinson’s patient’s tremor. It also serves as a complementary method to the B-SOiD unsupervised behavior segmentation platform launched two years ago.

Besides being an efficient program, A-SOiD is very affordable, can run on a regular computer, and is available as open source on GitHub.

A-SOiD is available to anyone interested in science

Jens Tillmann, a postdoctoral researcher at the University Hospital Bonn at the University of Bonn, said the idea of ​​making the program open to all researchers was part of its impact.

“This project would not have been possible without the clear scientific thinking that both our laboratories and the entire neuroethology community have demonstrated in recent years,” Tillmann said. “I am excited to be a part of this community and look forward to future collaborative projects with other experts in the field.”

Ittri and Martin K. Schwartz, principal investigators at the University Hospital Bonn and members of the interdisciplinary research areas (TRA) Life and Health at the University of Bonn, plan to use A-SOiD in their laboratory to further investigate the relationship between the brain and the brain. and behavior. Yttri plans to use A-SOiD in combination with other tools to investigate the neural mechanisms underlying spontaneous behavior. Schwartz will use A-SOiD in conjunction with other behavioral methods to analyze in detail known behaviors in social interactions.

Both Ittri and Schwartz expressed the hope that A-SOiD will be used by other researchers from different disciplines and countries.

“A-SOiD is a significant advance that allows us to delve into AI-based behavioral classification and thus represents a unique opportunity to better understand the causal relationship between brain activity and behavior,” Schwartz said. “We also hope that the development of A-SOiD will serve as an effective stimulus for future collaborative research projects focusing on behavioral research both in Europe and across the Atlantic.”