Apple researchers have released a new open-source artificial intelligence model that can organize images based on user instructions in natural language. It is called “MGIE”, which stands for MLLM-oriented image editing, and uses multimodal large language models (MLLM) to interpret user queries and perform pixel-level manipulations.

The model can edit various aspects of images. General photo enhancements may include brightness, contrast or sharpness, or applying artistic effects such as thumbnails. Local editing can change the shape, size, color, or texture of certain areas or objects in the image; Photoshop-style changes can include cropping, resizing, rotating and adding filters, or even changing backgrounds and blending images.

The user can specify a photo of a pizza to “make it healthier.” The model can add vegetable ingredients such as tomatoes and herbs using common sense. While a general optimization input request might be “add contrast to simulate more light,” a Photoshop-style change could be made by asking the model to remove people from the background of the photo by shifting the focus of the image. to the subject’s facial expression.

Apple collaborated with UCLA researchers to create MGIE, which was presented in a paper at the International Conference on Learning Representations (ICLR) 2024. The model is available on GitHub and includes code, data, and pre-trained models.

This is Apple’s second breakthrough in AI research in as many months. In late December, Apple announced that it had succeeded in deploying large language models (LLM) on the iPhone and other memory-constrained Apple devices by inventing an innovative technique for using flash memory.

For the past few months, Apple has been testing a rival called “Apple GPT” that could compete with ChatGPT. According to Mark Gurman BloombergWorking on artificial intelligence is a priority for Apple, as the company develops the “Ajax” framework for large language models.

AND Information, and analyst Jeff Poo claims that Apple will have some form of generative AI feature on “iPhone” and iPad when iOS 18 launches in late 2024. iOS 18 is said to include an improved version of Siri and have a similar generative AI feature. According to Gurman, ChatGPT may be the “biggest” software update in iPhone history.