[13:18 Tue,3.September 2024 by Thomas Richter] |
The newly released open-source text-to-video model, CogVideoX, has the potential to significantly accelerate the video AI revolution. This is because its code, along with its model weights, has been made available for free download at github.com/THUDM/CogVideo, and it can already run on commercially available graphics cards. As a result, any user with a reasonably powerful GPU on their home PC can generate videos using AI without worrying about costs due to excessive attempts. ;)
Competition for Runway, Luma, and Kling?Developed by researchers from Tsinghua University and Zhipu AI, CogVideoX generates videos up to 6 seconds long at a resolution of 720x480 pixels. The only downside is the low frame rate of 8 frames per second, which could, however, be increased afterward using intelligent interpolation via AI.
The first version of CogVideo was released back in 2022, but only the current version, CogVideoX, now delivers results that, according to researchers& benchmarks, are said to outperform competitors like Videocrafter 2.0 and Sora. However, everyone can form their own opinion based on the examples shown. Nonetheless, the quality bar for video AIs has been raised significantly, especially by the new Chinese model  MiniMax.
CogVideoX comes in two different versions, which differ in model size and therefore also in image quality and hardware requirements. The smaller model, CogVideoX-2B, already runs on older GPUs like the GTX 1080TI with 11 GB VRAM, and the larger CogVideoX-5B model, with 5 billion parameters, runs on mid-range GPUs like the RTX 3060 with 12 GB VRAM. With special parameters, the memory requirement of CogVideoX-5B can be further reduced so that it works on graphics cards with only 5 GB VRAM. Currently, however, only text-to-video generation is supported, with image-to-video or start/end keyframes (yet) to be implemented. The CogVideoX-2B model was released under the permissive Apache 2.0 license, while CogVideoX-5B is subject to the somewhat less permissive CogVideoX license, which allows users to use the software free of charge. However, for commercial use, a registration and free basic license are required, which permits commercial use with up to 1 million visits per month. Try it yourselfThe free demo of CogVideoX-5B on Hugging Face already integrates upscaling to 1,440 x 960 pixels and frame rate interpolation to 16 fps upon request. Several demo clips can also be found there. CogVideoX can also be installed on your own PC, provided you have a sufficiently powerful GPU with enough VRAM. ;) CogVideoX Depclips We predict that CogVideoX will have a significant impact on the development of video AIs due to the fact that its code is freely publicly available, making it easy to use and, above all, to further develop or integrate. The hardware requirements are so low that the barrier to entry is now very low since any gaming PC is sufficient. This ensures—similar to open-source LLMs like Meta&s Llama—that the further development and use of AI is not limited only to large corporations with the capital needed to train such models. Of course, this free availability of a high-quality video AI also increases the risk of even more DeepFake videos. It is not yet clear whether the researchers have implemented any precautions (e.g., restricting training data or prompt possibilities). But even if they have, the open-source nature of CogVideoX will likely soon invite circumvention of these measures.  more infos at bei github.comdeutsche Version dieser Seite: CogVideoX: Neue Open Source Video-KI läuft auf Consumer-GPUs |
 |
|
|