Aurally and Visually Enhanced Audio Search

Google Tech Talks
October 1, 2008

ABSTRACT

To create enhanced user interfaces for retrieval and processing of audio and other media, we developed a range of techniques that leverage the human brain's enormous capabilities in visual and auditory perception and draw on computer graphics as well as automated acoustic content analysis and new applications of human-computer interface devices. Our aim is to put more fun into these tasks--for both casual and professional users--and to provoke serendipity.

We demonstrate--among other examples--how to quickly search through large collections of sound effects by listening to several ones simultaneously, with the user being immersed in a spatial field of sounds. We show how to find music files with the help of procedurally generated icons that replace the standard file icons--a fleet-footed application of visual data mining for non-experts. Today, not only collections of media files can become overwhelming, but also collections of effect software can. To address this issue, we show how to represent a collection of audio plug-in effects as virtual folders through a FUSE system.

Speaker: Sebastian Heise
Sebastian Heise is a M.Sc. student in the Digital Media program of Hochschule Bremen (University of Applied Sciences), Germany. Sebastian has worked as a sound designer in game development teams at Gauselmann Group.

Speaker: Michael Hlatky
Michael Hlatky is a M.Sc. student in the Digital Media program of Hochschule Bremen (University of Applied Sciences), Germany. Michael has done research on sound quality for Bang a/s and AUDI AG.

Speaker: Jorn Loviscach
Jorn Loviscach is a professor in the Digital Media program of Hochschule Bremen (University of Applied Sciences), Germany. Jorn works on computer graphics, human-computer interaction, and audio and music computing. He is a regular contributor to conferences such as SIGGRAPH, Eurographics and the AES Convention. In addition, he has published numerous chapters in book series such as Game Programming Gems and ShaderX Programming.

Category:

People & Blogs

Tags:

• google
• techtalks
• techtalk
• engedu
• talk
• talks
• googletechtalks
• education

[lepetitzozo1]

Not sure I understood correctly the explanation of SoundTorch at the end, you calculate 13 MFCCs for several frames, and then you calculate the average of these 13 MFCCs accross the frames and that's what you are using for input vector for that sound in the self-organizing map?

[xerorxz]

This is amazing,my first impression was that this was of no real use.. but this looks like it really works....

[threelegduck]

how about smell?

[SpythonS]

Oh ok.. I was just wondering you don't have to give me thumbs down =(

[colddietcoke]

I think they actually mean Anally and Visually Enhanced Audio Search. Good catch!

[JuanRomeroDotCH]

I reckon not. They actually mean aurally as in: Of, relating to, or perceived by the ear.

[SpythonS]

You do mean :

Orally and Visually Enhanced Audio Search

right?