AudioLM

A new AI system can create natural-sounding speech and music after being prompted with a few seconds of audio. AudioLM, developed by Google researchers, generates audio that fits the style of the prompt, including complex sounds like piano music, or people speaking, in a way that is almost indistinguishable from the original recording. The technique shows promise for speeding up the process of training AI to generate audio, and it could eventually be used to auto-generate music to accompany videos. AI-generated audio is commonplace: voices on home assistants like Alexa use natural language processing. AI music systems like Open AI's Jukebox have already generated impressive results, but most existing techniques need people to prepare transcriptions and label text-based training data, which takes a lot of time and human labour. Jukebox, for example, uses text-based data to generate song lyrics. AudioLM, described in a non-peer-reviewed paper last month, is different: it doesn't require transcription or labelling. Instead, sound databases are fed into the programme, and machine learning is used to compress the audio files into sound snippets, called "tokens," without losing too much information. To generate the audio, a few seconds of sound are fed into AudioLM, which then predicts what comes next.