Stable Audio Open produces recordings of up to 47s

Stability AI, known for its AI-powered art generator Stable Diffusion, has just launched a new open AI model for sound and music generation called Stable Audio Open.

This new model has been trained exclusively on royalty-free recordings, expanding creative possibilities for musicians and audio producers.

Read too:

Unstable Diffusion, AI that generates pornographic content, creates more than 500 thousand erotic images per day
This tool for evaluating graphics card performance in Stable Diffusion costs $5,000

Stable Audio Open brings innovation to sound creation

Stable Audio Open is a generative model that transforms textual descriptions (such as “Rock beat played in a treated studio, session drums in an acoustic kit”) into recordings of up to 47 seconds. Trained with approximately 486,000 samples from free music libraries such as FreeSound and Free Music Archive, the model offers a wide range of possibilities for creating sounds.

According to Stability AI, Stable Audio Open can be used to create drum beats, instrument riffs, ambient noises and production elements for videos, films and TV shows. Furthermore, it is possible to “edit” existing songs or apply the style of one song to another.

One of the key benefits of Stable Audio Open, according to a Stability AI corporate blog post, is its open source nature, allowing users to fine-tune the model with their own custom audio data. “For example, a drummer can tweak the model with recordings of his own drums to generate unique new beats.”

Limitations and considerations of the new tool

Despite its innovative capabilities, Stable Audio Open has its limitations. It is not optimized to produce complete songs, melodies or high-quality vocals. For these purposes, Stability AI recommends its premium Stable Audio service.

Furthermore, commercial use of Stable Audio Open is prohibited by its terms of service. The model also presents uneven performance across different musical and cultural styles, in addition to having biases when used with descriptions in languages ​​other than English. Stability AI attributes these limitations to training data, which may lack cultural diversity.

“The data source possibly lacks diversity and all cultures are not equally represented in the dataset”Stability AI writes in the model description. “The samples generated by the model will reflect the biases of the training data.”

Controversies and future of AI in music

Stability AI has faced challenges stabilizing its business and was recently embroiled in controversy following the resignation of Ed Newton-Rex, the company’s vice president of generative audio. Newton-Rex disagreed with the company’s stance that using copyrighted works to train AI models constitutes “fair use.” The launch of Stable Audio Open appears to be an attempt to redefine that narrative while promoting the company’s paid products.

Admittedly, some AI-powered music generators, like those from Stability AI, are gaining more and more popularity. As a result, copyright issues are becoming increasingly central. In May, Sony Music, which represents artists including Billy Joel, Doja Cat and Lil Nas X, sent a letter to 700 AI companies warning against unauthorized use of their content to train audio generators. In March, the US state of Tennessee passed the first law aimed at cracking down on AI abuse in music.

Fonte: techcrunch