Midjourney Video Generator Announced – Can It Compete?

Artificial Intelligence

by Mascha DeikovaJanuary 16^th, 2024

The creators of Midjourney recently announced that their text-to-video generator will be released soon. According to CEO David Holz, AI model training will start in January, and the new tool should be introduced in a few months. It won’t be the first of its kind on the market, but we have many reasons to think it might become one of the best. And not only us! Let’s talk about why Midjourney for video is a long-awaited release.

Midjourney is an independent research lab that began its operations in February 2022 and has gained massive success ever since. Their AI creates images from text descriptions via so-called “prompts”, just like dozens of competitor’s tools – DALLE-3, and Adobe Firefly, to name a few. Although it still operates through a Discord bot, Midjourney is considered one of the leading image generators in the world. Users love its language understanding, dreamlike output, and amazing photorealistic results.

It’s a great tool for mood boards (read about our experience), or even for creating fancy zoom-out videos (here is a step-by-step guide). Soon, according to Midjourney, we will be using it for video previsualizations as well.

Midjourney for video as a natural development

Midjourney’s team rolls out updates quite frequently. Within a year of the fifth version, developers launched Midjourney V6 just before Christmas. It still lacks a lot of its predecessor’s functions and is considered to be an early alpha test, but even so, it is now crystal clear that this AI model will be able to create more realistic images than any of the previous releases. The update, which has been in the works for 9 months, also includes higher coherence, better sensitivity to the wording and understanding of the human language, and text drawing ability. For example, this is what I got as the result of the prompt “A long street at night, which disappears into fog. Everything is lit by a street lamp, and in the foreground, there is a sign with “2024” –ar 16:9 –v 6.0“:

To try out V6, you should type in “/settings” and choose it from the dropdown menu, or add the commend “–v 6” at the end of your prompt.

Generative AI experts out there may argue: “And? Other image generators introduced text creation ability months earlier!” Well, that’s the thing about Midjourney. They don’t try to be the pioneers in everything. A lot of features come out much later than the same ones from competitors (Vary (Region) vs Adobe’s Generative Fill is a good example). Yet, in most cases, they deliver much higher quality results. “Slower, but better” is also the approach Midjourney took in training their video generator. If the tool can produce videos with the same quality and coherency as Midjourney’s images, well, then the market will get some serious competition.

Is Midjourney late to the party of video generators?

As mentioned, video generation is not Midjourney’s terrain per se. Runway ML is currently running the show with their so-called AI magic tools (we touched on their Gen-1 and Gen-2 video models here). Google has worked on generating movies for more than a year. Also, there is Pika, which is still free of charge in the beta phase, and the freshest release from Stability AI – Stable Video Diffusion. Even the well-known image generator, Leonardo AI, recently incorporated an image-to-motion feature. (By the way, we will do a thorough overview of the video generative AI soon, so stay tuned!)

So, yes, with their announcement, Midjourney enters a crowded field. However, as we know, this tech is still very young and inexperienced, producing results that are far from perfect. Maybe Midjourney will be able to demonstrate another leap in AI development as they did with their image generator.

What would also be great to see from Midjourney is their take on the ethics question, as this research company (along with others) is often criticized for training their models on the works of artists without acquiring the rights for them.

Other changes are coming

The announcement about the upcoming Midjourney for video model came during the weekly office hours session on Discord. There are no further details about how it will work or when exactly to expect it.

Also, users can’t wait for other promised changes, like finally leaving Discord for good and having a better interface for Midjourney, which may also happen within the next months:

What do you think about the news? Are you excited to see Midjourney for video generation? Do you already use some other text-to-motion tools like Runway? Please, share your experience with us in the comments below!

Feature image: generated with Midjourney V6 for CineD