Musical Exploration of Volumetric Textures in Mixed and Virtual Reality

Florent Berthaut (1, 2)

1 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
2 MINT2 - Méthodes et outils pour l'Interaction à gestes

The development of technologies for acquisition and display gives access to a large variety of volumetric (3D) textures, either synthetic or obtained through tomography. They constitute extremely rich data which is usually explored for informative purposes, in medical or engineering contexts. We believe that this exploration has a strong potential for musical expression. To that extent, we propose a design space for the musical exploration of volumetric textures. We describe the challenges for its implementation in Virtual and Mixed-Reality and we present a case study with an instrument called the Volume Sequencer which we analyse using your design space. Finally, we evaluate the impact on expressive exploration of two dimensions, namely the amount of visual feedback and the selection variability.

Keywords: 3D textures; Virtual reality; Mixed reality; Musical exploration

Domains: Sound [cs.SD]; Human-Computer Interaction [cs.HC]; Musicology and performing arts; Graphics [cs.GR]

Florent Berthaut. Musical Exploration of Volumetric Textures in Mixed and Virtual Reality. International Conference on New Interfaces for Musical Expression (NIME 2021), Jun 2021, Shangai, China. ⟨10.21428/92fbeb44.6607d04f⟩. ⟨hal-03260466v2⟩