Meta releases AI music model MusicGen

2023-06-13

Compared to other music models such as Riffusion, Mousai, MusicLM, and Noise2Music, MusicGen performs better in both objective and subjective measures. Meta's MusicGen is an AI model that uses the Transformer architecture to generate new music compositions based on textual prompts. It has the ability to maintain consistency with existing melodies, providing a versatile and innovative approach to music creation. Similar to language models, MusicGen predicts the next section of a music composition rather than the next character in a sentence. This allows it to generate coherent and structured music compositions. The training process involves using Meta's EnCodec audio tagger to break down the audio data used for training into smaller components. This approach enables the model to parallel process tags, making it efficient and fast in generating music. The training process involves a dataset of 20,000 hours of licensed music, including 10,000 high-quality music tracks from an internal dataset, as well as music data from Shutterstock and Pond5. This extensive training dataset ensures that MusicGen has access to a wide range of music styles and compositions. One key feature of MusicGen is its ability to handle both textual and musical prompts. Textual prompts set the basic style and then match it with the melody in an audio file. For example, by combining a textual prompt describing a specific music style with the melody of a famous composition, MusicGen can generate new music that reflects the desired style. It is important to note that while MusicGen can provide a rough guidance for generating music based on specific prompts, it cannot precisely control the direction of the melody or the ability to hear melodies in different styles. The generated output is a creative interpretation rather than a perfect replica. In terms of performance, researchers experimented with models of different sizes, ranging from 300 million to 3.3 billion parameters. They found that larger models generally produce higher quality audio, but the model with 1.5 billion parameters was rated as the best by human evaluators. The model with 3.3 billion parameters performed exceptionally well in accurately matching text inputs and audio outputs. Compared to other music models such as Riffusion, Mousai, MusicLM, and Noise2Music, MusicGen outperforms them in objective and subjective measures such as the match between music and text and the credibility of composition. Overall, MusicGen ranks higher than Google's MusicLM and may be the StableDiffusion moment in the music industry. Meta has released the code and models of MusicGen on GitHub in an open-source manner, allowing researchers and commercial users to use and leverage this technology. This move encourages further development, collaboration, and innovation, driving the advancement of AI-generated music. Additionally, the functionality of MusicGen can be experienced on the Huggingface platform. Source: https://analyticsindiamag.com/meta-releases-stablediffusion-for-music-musicgen/

RECENT AI NEWS

RECENT AI TOOLS