GUESS : GradUally Enriching SyntheSis for Text-Driven Human Motion Generation

In this article, we propose a novel cascaded diffusion-based generative framework for text-driven human motion synthesis, which exploits a strategy named GradUally Enriching SyntheSis (GUESS as its abbreviation). The strategy sets up generation objectives by grouping body joints of detailed skeleton...

Ausführliche Beschreibung

Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on visualization and computer graphics. - 1996. - 30(2024), 12 vom: 31. Okt., Seite 7518-7530
1. Verfasser: Gao, Xuehao (VerfasserIn)
Weitere Verfasser: Yang, Yang, Xie, Zhenyu, Du, Shaoyi, Sun, Zhongqian, Wu, Yang
Format: Online-Aufsatz
Sprache:English
Veröffentlicht: 2024
Zugriff auf das übergeordnete Werk:IEEE transactions on visualization and computer graphics
Schlagworte:Journal Article
LEADER 01000caa a22002652 4500
001 NLM367150565
003 DE-627
005 20241029231940.0
007 cr uuu---uuuuu
008 240116s2024 xx |||||o 00| ||eng c
024 7 |a 10.1109/TVCG.2024.3352002  |2 doi 
028 5 2 |a pubmed24n1584.xml 
035 |a (DE-627)NLM367150565 
035 |a (NLM)38224502 
040 |a DE-627  |b ger  |c DE-627  |e rakwb 
041 |a eng 
100 1 |a Gao, Xuehao  |e verfasserin  |4 aut 
245 1 0 |a GUESS  |b GradUally Enriching SyntheSis for Text-Driven Human Motion Generation 
264 1 |c 2024 
336 |a Text  |b txt  |2 rdacontent 
337 |a ƒaComputermedien  |b c  |2 rdamedia 
338 |a ƒa Online-Ressource  |b cr  |2 rdacarrier 
500 |a Date Completed 28.10.2024 
500 |a Date Revised 28.10.2024 
500 |a published: Print-Electronic 
500 |a Citation Status MEDLINE 
520 |a In this article, we propose a novel cascaded diffusion-based generative framework for text-driven human motion synthesis, which exploits a strategy named GradUally Enriching SyntheSis (GUESS as its abbreviation). The strategy sets up generation objectives by grouping body joints of detailed skeletons in close semantic proximity together and then replacing each of such joint group with a single body-part node. Such an operation recursively abstracts a human pose to coarser and coarser skeletons at multiple granularity levels. With gradually increasing the abstraction level, human motion becomes more and more concise and stable, significantly benefiting the cross-modal motion synthesis task. The whole text-driven human motion synthesis problem is then divided into multiple abstraction levels and solved with a multi-stage generation framework with a cascaded latent diffusion model: an initial generator first generates the coarsest human motion guess from a given text description; then, a series of successive generators gradually enrich the motion details based on the textual description and the previous synthesized results. Notably, we further integrate GUESS with the proposed dynamic multi-condition fusion mechanism to dynamically balance the cooperative effects of the given textual condition and synthesized coarse motion prompt in different generation stages. Extensive experiments on large-scale datasets verify that GUESS outperforms existing state-of-the-art methods by large margins in terms of accuracy, realisticness, and diversity 
650 4 |a Journal Article 
700 1 |a Yang, Yang  |e verfasserin  |4 aut 
700 1 |a Xie, Zhenyu  |e verfasserin  |4 aut 
700 1 |a Du, Shaoyi  |e verfasserin  |4 aut 
700 1 |a Sun, Zhongqian  |e verfasserin  |4 aut 
700 1 |a Wu, Yang  |e verfasserin  |4 aut 
773 0 8 |i Enthalten in  |t IEEE transactions on visualization and computer graphics  |d 1996  |g 30(2024), 12 vom: 31. Okt., Seite 7518-7530  |w (DE-627)NLM098269445  |x 1941-0506  |7 nnns 
773 1 8 |g volume:30  |g year:2024  |g number:12  |g day:31  |g month:10  |g pages:7518-7530 
856 4 0 |u http://dx.doi.org/10.1109/TVCG.2024.3352002  |3 Volltext 
912 |a GBV_USEFLAG_A 
912 |a SYSFLAG_A 
912 |a GBV_NLM 
912 |a GBV_ILN_350 
951 |a AR 
952 |d 30  |j 2024  |e 12  |b 31  |c 10  |h 7518-7530