Be a part of our day by day and weekly newsletters for the most recent updates and unique content material on industry-leading AI protection. Be taught Extra
ByteDance researchers have developed a synthetic intelligence system that transforms single pictures into practical movies of individuals talking, singing and transferring naturally — a breakthrough that might reshape digital leisure and communications.
The brand new system, known as OmniHuman, generates full-body movies displaying individuals gesturing and transferring in ways in which match their speech, surpassing earlier AI fashions that might solely animate faces or higher our bodies.
How OmniHuman Makes use of 18,700 Hours of Coaching Information to Create Reasonable Movement
“Finish-to-end human animation has undergone notable developments lately. Nonetheless, present strategies nonetheless battle to scale up as giant common video technology fashions, limiting their potential in actual functions,” the researchers wrote in a paper printed on arXiv.
The crew skilled OmniHuman on greater than 18,700 hours of human video knowledge utilizing a novel strategy that mixes a number of kinds of inputs — textual content, audio, and physique actions. This “omni-conditions” coaching technique permits the AI to study from a lot bigger and extra various datasets than earlier strategies.
AI video technology breakthrough exhibits full-body motion and pure gestures
“Our key perception is that incorporating a number of conditioning alerts, corresponding to textual content, audio, and pose, throughout coaching can considerably cut back knowledge wastage,” the analysis crew defined.
The know-how marks a major advance in AI-generated media, demonstrating capabilities from creating movies of individuals delivering speeches to displaying topics enjoying musical devices. In testing, OmniHuman outperformed present programs throughout a number of high quality benchmarks.
Tech giants race to develop next-generation video AI programs
The event emerges amid intensifying competitors in AI video technology, with corporations like Google, Meta, and Microsoft pursuing comparable know-how. ByteDance’s breakthrough may give the TikTok father or mother firm a bonus on this quickly evolving area.
Trade consultants say such know-how may rework leisure manufacturing, instructional content material creation, and digital communications. Nonetheless, it additionally raises issues about potential misuse in creating artificial media for misleading functions.
The researchers will current their findings at an upcoming laptop imaginative and prescient convention, although they haven’t but specified which one.
