Developed by Cornell researchers, the app uses augmented reality to help users repeatedly take photos from the same location with a phone or tablet to create time-lapse videos without leaving the camera in place. Time-lapse photography, which involves combining photos taken over a long period of time, offers a powerful way to visualize events such as changing seasons or the movement of the sun.

Traditionally, photographers left the camera on a tripod for the duration of the event, but researchers working with Abe Davis, an associate professor of computer science at Cornell’s Ann S. Bowers College of Computer and Informatics, developed a more convenient method. The ReCapture iOS app is now available for free on the Apple App Store.

Researchers believe this is the first program designed to create slow motion videos from portable devices.

Ruyu Yan, a computer scientist at Cornell Engineering, the program’s lead developer, presented the paper “ReCapture: AR-Guided Slow-Motion Capture” at the 2022 Association for Computing Machinery (ACM) User Interface Software Symposium. and technology on Nov.

The app has three shooting modes covering various scenarios. One is best for landscapes, one helps capture close-up scenes, and the third collects a set of images that can be used to recreate the scene in 3D offline.

Each shooting mode uses different scene information. The simplest mode uses overlaying previous shots to help the user create new photos. For close-up scenes that are more difficult to capture, the app tries to determine where the camera is in 3D space and uses arrows to guide the user how to move and tilt their phone in the correct location direction.

This work emerged from Ian’s summer research with Davis as part of the Computer Science Research Program. Ian mentioned that the participants were interested in geocaching, an activity in which they use GPS to find a box of junk, called cache, hidden by other geocaching enthusiasts. Meanwhile, Davis envisioned a project that would help field researchers repeatedly locate and re-photograph precise locations on field plans to monitor for any changes. Together they came up with the idea of ​​”geocaching with images,” which eventually became ReCapture.

“Geocaching is something people do for fun, but if you’re a scientist and you’re doing fieldwork, you have a similar problem,” Davis said.

Jiatian Sun, a PhD student in computer science, and Longxiulin Deng, a computer science major at Cornell Engineering, also helped with the research. The hardest part is designing the app’s interface to guide users through the process because “what works intuitively for me may not intuitively work for others,” Yang said. He sought feedback from 20 beta testers and worked with the XR Collaboratory at Cornell Tech, which advised researchers on augmented reality, virtual reality, and mixed reality applications.

Additionally, he needed to figure out how to manage mountains of data associated with the photos. “The program used to crash a lot,” he said. This was a problem because if the app was too slow or crashing constantly, people weren’t getting enough footage, resulting in choppy, low-quality videos.

According to Davis, future versions of the program will be able to correct gaps and abrupt transitions in video material using the latest machine learning methods, which will enable higher quality videos.

As Davis envisioned, the program could have valuable scientific applications beyond creating gifs and videos. The team shared the app with field researchers from other departments at Cornell University, and colleagues at the School of Integrative Plant Science in the College of Agriculture and Life Sciences have already started using the app to collect data. Source