Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, Artsiom Sanakoyeu
International Conference on Immersive and 3D Audio (I3DA)
Binaural reproduction over headphones is becoming popular with applications in teleconferencing, virtual reality and entertainment. Many binaural reproduction methods are developed and studied extensively for spherical and other specially-designed arrays. However, some applications require simpler arrays, such as wearable or mobile. Recently, a binaural reproduction method was developed for such arrays based on binaural signals matching (BSM). While showing that BSM can produce accurate binaural signals, the study was limited to two array configurations and head-tracking was not incorporated. As a follow-up, this paper studies BSM with a varying number of microphones in a semi-circular configuration and investigates a solution for head-tracking. Simulations show that placing two microphones close to the ear positions of the head related transfer functions significantly improves reproduction with a static head. However, microphones placed away from ear positions are needed for head-tracking. This is then subjectively studied in a listening experiment with a semi-circular array, showing that compensating for head-rotations is important for accurate spatial perception but may introduce timbral artifacts due to the limited number of microphones.
Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, Artsiom Sanakoyeu
Bilge Acun, Benjamin Lee, Fiodar Kazhamiaka, Kiwan Maeng, Manoj Chakkaravarthy, Udit Gupta, David Brooks, Carole-Jean Wu
Ilkan Esiyok, Pascal Berrang, Katriel Cohn-Gordon, Robert Künnemann