We are seeing a steady increase in the development of generative artificial intelligence (AI) such as ChatGPT and DALL-E based on OpenAI. But Runway excels at AI R&D focused on editing and content creation, which was part of this expressive group of machines.

Its main products are the Gen-1 and Gen-2 generative AI models, released at the end of March, which attracted attention with the ability to create short videos using only text commands, resulting in a very pleasant content for users. these tools.

In this article, we will understand how Gen-2 works, comment on the results of our tests in the free version, and analyze the future possibilities of this technology in the generation of creative content.

How it works and tests

First, some terms need to be explained, starting with generative AI, which are systems capable of creating new content from existing reference databases, which can be images, video, audio, or text.

The statistical model that these AIs typically use is the distribution model, which works through an iterative process of adding and removing noise from information that incrementally improves content creation until the desired result is achieved.

Within diffusion, there is also stable diffusion, a technique used to gradually train and update AI models, avoiding abrupt changes that can lead to instability or significant performance fluctuations.

Both Gen-1 and Gen-2 use these methods to detect, treat and achieve incredible results. The company already has a free iOS app, RunwayML, where users can now use these tools on their smartphones, demonstrating that the barriers to AI creativity are no longer limited to desktops!

It should be noted that two types of subscription are available to use 100% of the capabilities of the company’s programs: monthly and annual. Both options also have a Standard and Pro type, both give access to more editing options and free up more storage space; The standard cost is $15 and $12 per month and year, and the Pro is $35 and $28.

Runway itself bills itself as “shaping a new era of art, entertainment and human creativity” and does so through software that helps audiovisual professionals and curious Internet users create content more easily.

These tools are built to be easy to use, as the company seeks to dispel the idea that people need prior experience in the field and robust platforms to create more complex content with AI.

Currently, there are other AIs that create image/video/text for video, but nothing compares to what the company has created!

To find out if the company managed to create a really easy to use program, we tested their web program. And the results, as simple as they may seem to some, are incredible for reasons that will be discussed later. Let’s move on to the tests.

Test 1: “The course of the Amazon River at dawn.”

Generative AI already knows how to create stunning landscapes, and Gen-2 didn’t disappoint when it created an aerial scene of dawn on the Amazon River. Lighting, reflections and sky are the most impressive parts of this creation. He also inserted some kind of fog over the river and some waves in the corner of the screen. Will it be Pororoka?

Test 2: “Street scene in São Paulo, Brazil at noon.”

Now we’re trying a more complex scenario where he created a street with low foot traffic and a mix of building styles, which is very typical of a city. All that was missing was the tangled wires and midday chaos in São Paulo.

Joking aside, the Gen-2 again managed to achieve incredible lighting and reflections, where it also did a great job with shading and very realistic skies.

Other details that are interesting to note are that in the bottom left of the video, the “camera” captures the inside of the building, but because it’s too dark, it only captures reflections of what appears to be a doorframe. and interior lighting, it’s amazing!

Test 3: “Sunny day scene on a beach in Brazil.”

There was a contrast of realism and weirdness in this test, especially when it came to the human anatomy in the video. What little he shows on people’s faces and hands is a bit strange, plus the food on the table is quite distorted.

Important note: This prompt did not ask for people or food to be inserted, Gen-2 had “creative” freedom to implement the image with these elements.

Test 4: “Medieval city with many citizens, flags decorating the city, a few trees on the streets, against the backdrop of a large golden castle.”

Here we decided to ask for more details and see how Gen-2 fared, but it ended up giving lower results than the other clues as some elements were much more accentuated than others. He also did not provide other requested elements, such as trees and a golden castle, as “many citizens” were in the background.

The sky, some lighting cases and the overall composition were not as satisfying as the previous ones, which shows us that the AI, as expected, still has room for improvement. It’s nice to note that Gen-2 has already managed to reproduce some very impressive cloth movement and lighting mechanics!

Test 5: “Space war in orbit of the red planet with large ships and laser beams.”

Finally, the most interesting result was the space battle intro, which managed to create a scene with quite rich colors and a bright background. The absence of a red planet and a more geometrically accented spaceship do not reduce the quality of the video.

Usage lens flare in lasers, the addition of other stars and a few ships in the background again shows Gen-2’s ability to create a continuous scenario that sometimes exceeds expectations.

Final thoughts

Due to the recent emergence of AIs that create fantastic and/or very realistic art, some people are becoming less and less impressed with the results and sizes that these tools have taken on.

But Gen-2 cannot be ignored, as this model has shown quite impressive results; the icing on the cake is the “mastery” of some representations of reality, such as the already mentioned lighting and fluid mechanics.

Runway has managed to create a tool with an incredibly promising ability to generate content for the generative art market, further opening up a horizon that was already fruitful. And of course, like all AIs, Gen-2 is in a state of constant learning, and with use and user feedback, it will remain on the radar as a constantly challenging program for its competitors.

Among the AIs that have already been tested here by the portal, this one easily ranks among the top five most amazing!

Source: AI Magic Tools Gen-2, Streaming Templates, Stable Streaming, Introducing Gen-2: Text to Video, Introducing: Runway for Mobile, Gen-2 by Runway