Realtime Generation of Audible Textures Inspired by a Video Stream

Authors

  • Simone Mellace Dalle Molle Institute for Artificial Intelligence
  • Jérôme Guzzi Dalle Molle Institute for Artificial Intelligence
  • Alessandro Giusti Dalle Molle Institute for Artificial Intelligence
  • Luca M. Gambardella Dalle Molle Institute for Artificial Intelligence

DOI:

https://doi.org/10.1609/aaai.v33i01.33019865

Abstract

We showcase a model to generate a soundscape from a camera stream in real time. The approach relies on a training video with an associated meaningful audio track; a granular synthesizer generates a novel sound by randomly sampling and mixing audio data from such video, favoring timestamps whose frame is similar to the current camera frame; the semantic similarity between frames is computed by a pretrained neural network. The demo is interactive: a user points a mobile phone to different objects and hears how the generated sound changes.

Downloads

Published

2019-07-17

How to Cite

Mellace, S., Guzzi, J., Giusti, A., & Gambardella, L. M. (2019). Realtime Generation of Audible Textures Inspired by a Video Stream. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 9865-9866. https://doi.org/10.1609/aaai.v33i01.33019865

Issue

Section

Demonstration Track