Dance Generation by Sound Symbolic Words

2023. 11. 04

This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience.



ユーザー調査によってオノマトペとダンスモーションのデータセットを作成し、ダンスモーション生成モデルであるAI Choreographerと、音素と音節に着目したオノマトペの印象数値化システムを使用し、オノマトペの音象徴性とダンスモーションの対応を学習した。


Miki Okamura *1     Naruya Kondo *1    Tatsuki Fushimi *1    Maki Sakamoto *2    Yoichi Ochiai *1

*1 University of Tsukuba    *2 The University of Electro-Communications