Bytedance/TikTok show the latest AI clip generator with motion control

[12:12 Mon,12.February 2024 by Rudi Schmidts]

Boximator is an AI model from Bytedance Research - i.e. from TikTok&s business environment. Like many of its predecessors, this new tool is used to generate realistic and controllable movements in AI video synthesis.

Traditionally, AI video synthesis is controlled by a text prompt ("text to video"). Recently, a demonstrative image can also be added to serve as a source image. An AI model then generates a moving video clip from this input. However, as it is not always possible to describe everything exactly with text, or many possibilities remain ambiguous, the result often does not meet the user&s expectations.

Bytedance/TikTok show current AI clip generator with motion control

Boximator takes a new approach to input: boxes or paths can be used to show the model exactly what movement you want in the image. This is not groundbreakingly new, but the results of the invented movements look state of the art. The legs of walking people often appear to be very well done (which is usually still a problem with other current AI models). The following video gives a good overview of how it works and the quality of the results:

A demo website will be published in 2-3 months so that you can try out Boximator with your own data. If you can&t wait any longer, you can also get in touch with the team by email to try out the tool immediately. More details can be found (at the bottom of the linked Github project page). The underlying AI model PixelDance

can be found here.

A

Arxiv paper is also already available, but there is no public code for the project yet.

more infos at bei boximator.github.io

deutsche Version dieser Seite: Bytedance/TikTok zeigen aktuellen KI-Clipgenerator mit Bewegungskontrolle