Lightricks LTX Studio – AI Video Generator with Most Extensive ‘Manual’ Controls Yet

Artificial Intelligence

by Omri Keren LapidotFebruary 29^th, 2024

Famous for their clever and innovative AI-based filters, Lightricks has come up with something that even sci-fi hardly predicted. The visual tech company just redefined the world of AI-generated video content by releasing their LTX Studio. Under the rather dramatic slogan “storytelling transformed” LTX Studio is an AI engine, but unlike its current peers, it provides a level of control we’ve never seen before.

The justified buzz around OpenAI’s SORA still lingers as Lightricks ups the game with their revolutionary new tool – LTX Studio. While SORA seems to be more realistic and visually refined at first glance, LTX Studio possesses a much more significant trait. It gives the user unprecedented control over the final video.

LTX Studio solves one of the more prominent problems of AI-based content creation – selective control. At its core, the creative process follows the basic rule of AI generators: Text to video. You simply type a text specifying your creative vision and the machine spews out a video. Most generators won’t go much further. You’ll be able to repost the prompt and the AI will generate a new video, but precise or selective control is a pretty rare feature. At this point LTX Studio parts away from their peers, letting you specifically select, edit, or add multiple items.

Firestarter for each visual process

So you have your general idea for your next blockbuster. Once you type in some relevant keywords, the LTX Studio will turn them into a rudimentary script, and once this script is fed into the AI, it will generate a video based on it. This part alone can speed up every ideation or inspirational process, but it’s what comes next that makes this system unique because, unlike most current AI generators, you’ll get more than a video. You’ll get footage to work with.

As you can see in this early workspace screenshot, the video is split into cuts with a specific synopsis for each. These cuts may be switched, replaced, cut, or expanded (by generating continuous “footage”).

Complete editing suite

LTX Studio provides you with a complete editing suite. Music, VFX, voice-over, cuts, and transitions will all be available to fine-tune in the final video. This feature (and others – so keep reading) makes the LTX Studio potentially revolutionary, and sets it aside from the AI crowd. But there’s more.

Character consistency

Character consistency stands as one of the major hurdles in the Turing tests of our time. AI generators seem to struggle with this kind of continuity, creating ever so slightly different character renderings that are highly apparent to the human eye. LTX Studio claims to have this issue covered. The system allows for the generation of distinct characters that encompass both appearance and personality traits, aiming to emulate characters that feel more relatable and “authentic.”

3D perception

The LTX Studio cam also “perceives” three-dimensional objects. While not a 3D application per se, this perception allows for some interesting features. The user will be able to control camera movement and angles, characters or object movement, and lighting positions. This also allows for different styling, atmosphere, ambient lighting conditions, and other modifications while keeping the main characters unchanged.

Casting

Just as we can change the ambient atmosphere, every character can be changed, edited, and fine-tuned to perfectly fit the creative vision. You can have the exact same scene and then switch the character’s hairdo, clothes, or makeup. But why stop there? Ethnicities, gender, height, or species – all are subject to your will.

It seems like the possibilities are limitless, which they probably are. We are, of course, far from covering every one of them in this article, and as always with AI, it’s just the beginning. However, as powerful as this new tool is (and IT IS), there still are some shortcomings.

Still missing

As of now, the video renders coming out of the LTX Studio bear the visual signature associated with AI-generated content. They don’t look as (scaringly) refined as SORA’s output, but will definitely hold in various use cases. Furthermore, it seems the AI can’t quite generate speech. It does away with voice-over, but I haven’t seen a character talk with lip-sync and vocal audio to match. There are also some issues with hands and with “unsynced” motion (when, for example, a man moves faster than his feet, “sliding” on the pavement, etc.). As testing ensues, we’ll probably spot more shortcomings, but those are vastly outweighed by the disruptive nature of this product.

What does the future hold?

At this point, I wouldn’t dare to prophesize. Last month brought such exorbitant advancement to this field, and the same goes for the last couple of years. Perplexing times indeed. I’ll wait to see if and how Lightricks incorporates this powerful tool into other existing (and future) tools. Also – we’ll see if the company chooses to work with authentication initiatives such as C2PA. Imagine editing your first feature film or independent documentary with these kinds of tools. Imagine readjusting the camera or lighting in post, or testing different styles and vibes on the same scene. Imagine adding those few extra frames based on your own footage so it perfectly aligns with the music. Imagine adapting the same commercial video to different audiences at the click of a button. What a time to be alive. LTX Studio has a waiting list open for requests.

Do you believe tools like this will take over visual motion content and filmmaking in years to come? How are you preparing yourself for the foreseeable future in this regard? Let us know in the comments.