[12:03 Mon,25.November 2024 by Thomas Richter] |
Lightricks has introduced a new video AI called LTX Video, along with its source code. This AI can run on home PCs equipped with a powerful graphics card like the Nvidia 4090 and is the first open-source video model capable of generating video clips in real time. The new video model, with just 2 billion parameters, generates 5-second videos at a resolution of 768x512 pixels and 24 frames per second in just 4 seconds using image-to-video or text-to-video input. This speed is even slightly faster than real time—but achieving this requires a Nvidia H100 GPU. On a Nvidia 4090, the generation takes about 20 seconds. LTX Video is available as a preview with ComfyUI integration on Hugging Face, GitHub, and Fal.ai. It can be downloaded or tested online by anyone, provided the system has at least 12 GB of VRAM. For example, on a Nvidia 3060, generating a clip takes approximately 80 seconds. The open but academic license for the preview version currently excludes commercial use, but the final version is expected to be free for both private and commercial use.
Speed vs. QualityThe high speed of LTX Video comes at the cost of reducing the parameter count of the AI model, often resulting in suboptimal outcomes. On GitHub, there is a collection of reasonably good-looking demo clips, but they appear to have been carefully selected—users report some bizarre results with self-generated clips. The quality seems to depend heavily on the chosen subject/prompt, with complex motions like fighting or dancing causing particular difficulties. The image-to-video model seems to produce better results than text prompts.
Other free video AIs like PyramidFlow, CogVideoX, or Genmo Mochi 1 appear to deliver much better quality but are significantly slower. Therefore, those looking to generate videos on their PC for free now have a choice between multiple higher-quality video AIs and LTX Video, whose extremely fast generation time allows for many attempts in the same timeframe. Hopefully, the final model will also see further improvements. LTX Video Examples LTX StudioWe had already reported last March on LTX Studio, developed by Lightricks, a comprehensive browser-based solution for filmmaking via AI that provides all the functionality for film production under a single interface—from story development to (pre-)production to post-production. LTX Studio integrates AI tools for story development, storyboarding, casting (meaning consistent character generation in this context), sound effects, voiceover, relighting, titling, and automatic editing, as well as video generation as its foundation. §PIC3:LTX Studio LTX Studio uses a mix of open-source models, licensed third-party services, and proprietary technology for its various functions. It has been publicly accessible since August. So far, the AI videos created with LTX Studio, with their maximum 4-second-long moving images, resemble animated slideshows more than actual video and still exhibit significant morphing artifacts. This is expected to change once the final version of LTX Video is integrated into LTX Studio. Bild zur Newsmeldung: deutsche Version dieser Seite: LTX Video - Neue Open Source KI generiert Videos schneller als in Echtzeit |