"Beethoven AI" Born?

2019-10-14

"Music is art, and music is also a business. Nowadays, the music business has attracted the attention of commercial giants. In the past, music platforms competed in terms of the number of tracks and the number of artists, but later they focused more on exclusive copyrights. Now, AI technology represented by deep neural networks is gradually approaching implementation, and the AI battle in the music platform industry is about to begin. In 2016, Google Brain launched the Magenta project, which started with the NSynth neural network audio synthesis algorithm and now includes the Coconet machine learning model that can restore Bach music from incomplete fragments. These projects are more focused on academic research. Sony, one of the world's three major music copyright companies, has a significant advantage in music content itself. In 2016, Sony Computer Science Laboratory introduced Flow Machines, a large-scale song and style database, and created melodies in the style of "The Beatles". Both Google and Sony have their own streaming products, such as YouTube Music and Sony Hi-Res. Similar to each other, streaming services may disrupt the traditional record companies and music copyright agencies, and they are bound to occupy a new industry collaboration model. The difference is that Sony needs to find the next growth point in the music copyright business after its revenue slows down. With the popularity of smart speakers, Google can also use its own smart speaker, Home, as the core to drive the smart home ecosystem. However, relying on low prices to attract consumers to smart speakers is not ideal, and the future ultimately depends on smart interactions and the coverage of the entire scene. Microsoft's "Xiaoice" targeting the Chinese market has now grown to its seventh generation. Based on the Avatar Framework AI framework, it not only focuses on intelligent dialogue and voice interaction but also specializes in simulating human voices and composing lyrics. In 2018, Microsoft Xiaoice proposed a semi-open ecosystem strategy called Dual AI and reached platformization agreements with several domestic companies, but it has not yet outlined a clear business logic. ByteDance, which is ambitious in the overseas market, has acquired Jukedeck, a startup company, and obtained the music copyrights of two major Indian record companies, T-Series and Times Music, in an attempt to use neural networks to synthesize music in its TikTok short video product. Short videos provide an effective way to achieve large-scale music composition through AI, which may alleviate the pressure on music copyrights. Currently, Google, Sony, Microsoft Xiaoice, and ByteDance are all fully deploying AI, but their progress in AI music varies. Early researchers focused more on making computers imitate existing music fragments and creating music melodies by analyzing the patterns in them. The difference in AI music composition is that it allows computers to truly create relatively complex and story-rich music by learning from a large number of music fragments. Google and Sony started exploring AI music composition early on. In contrast, Jukedeck, which was acquired by ByteDance, is still in the imitation stage and can only be used as a mass production tool in the music production line. In a sense, technological advancements are driving progress in the music industry. From the early CD era to today's AI music, the production, distribution, and consumption of music have undergone significant iterations and upgrades. According to the International Federation of the Phonographic Industry (IFPI) report, the global music market revenue in 2018 increased by 9.7% compared to the previous year, reaching $19.1 billion. For technology giants seeking deep-level business evolution, the key to victory ultimately depends on seizing opportunities. More importantly, the AI-driven music market revolution will ignite a new round of competition with the frequent actions of these giants. AI is reshaping the global competitive landscape of enterprises, and the music industry has also entered a critical period of activating diversified value through AI. Nowadays, AI has demonstrated its strength in the field of music far beyond people's imagination. The world's first AI symphonic variation, "Me and My Motherland," was premiered by the Shenzhen Symphony Orchestra on October 11. It was produced by Ping An Artificial Intelligence Research Institute in China. This work uses Ping An's AVM automatic variation system, based on the massive historical music works data accumulated by the Ping An AI team, systematic music label engineering, systematic automatic variation, music evaluation, and expert rule models. Through the fusion of deep learning and reinforcement learning AI technologies, it created an AI symphony composition covering five variation sections based on classic tracks such as "Me and My Motherland" and "In the Field of Hope." The composition portrays the magnificent development history of the Chinese nation from the Opium War to the present. The symphony lasts for 12 minutes and features various tones such as organ and violin, as well as different voices including the main melody, accompaniment, and percussion beats. It is very rich overall, and ordinary people cannot tell that it is an AI-composed piece. It sounds like the work of a professional human musician. The creation of the AI symphonic variation of "Me and My Motherland" took four months and involved the training of over 700,000 songs. Unlike current AI compositions that mostly focus on single melodies and short compositions, Ping An's attempt is the world's first AI-composed symphony work with multiple voices, broad dimensions, complexity, and classical heritage. The creation process involved the use of multiple technical models and the development of the AVM automatic variation model based on this symphony. Specifically, Ping An used a database of historical music works and a systematic music label engineering based on deep learning and reinforcement learning AI technologies. It utilized automatic variation models, music evaluation models, and expert rule systems to analyze the combination space of musical notes, select the best music segments, and complete the composition. As the foundation for AI composition learning, Ping An has built six databases, including a song library, composition rule library, lyrics material library, music comment library, vocal source library, and instrument source library, which encompass millions of composition materials. For the creation of this AI symphonic variation, over 700,000 songs were used for structured training, including classical music, revolutionary songs, folk songs, and other genres. At the same time, the composition team set up a music label system based on classical composition theories, creating a massive dimension music label system that includes emotions, styles, themes, development techniques, harmony, forms, counterpoints, orchestrations, tonality, modes, and time signatures. It is worth noting that Ping An has also developed an automatic annotation analysis system based on manually selected datasets, attempting to apply AI technology to automatically analyze music audio. In terms of model application, the AI symphonic variation of "Me and My Motherland" uses Ping An's AVM automatic variation model. First, it constructs a basic model through the construction of expert variation rules in rhythm, harmony, texture, and orchestration. Then, a combination of deep learning and reinforcement learning is used to train the basic model based on the description rules of music composition theory, enabling the machine to grasp the thinking logic of human composition and learn the techniques of musical variation. The music evaluation model, which is trained through a large number of works by composers, is also deeply learned and reinforced. It is known that the evaluation criteria for whether a piece of music is pleasant are relatively subjective, but the rules of composition have gradually been established in the development of music, forming relatively objective industry standards. Therefore, in the process of music creation, it is necessary to comply with the mainstream aesthetics and consider the evaluation standards of composition experts while adhering to the principle of selecting the best music segments. To prevent AI composition from being too free, Ping An incorporates expert rules, including harmony constraints, counterpoint constraints, and form structure constraints, into the process of AI music composition, allowing AI composition to closely resemble the original genre of the music and possess the classic nature of the times. Overall, the birth of the AI symphonic variation of "Me and My Motherland" not only explores the technological, structural, and audio analysis aspects based on AI but also relies on professional music understanding. It achieves deep learning of excellent works in human history and trains AI models. The Ping An AI composition team continuously attempts to migrate AI algorithms to generate various styles and types of variations, evaluates these variations professionally, and finally presents them through large-scale orchestras. The fusion of AI creation and human interpretation signifies a significant advancement of artificial intelligence technology in the field of art, continuously breaking through technological boundaries. We have also seen that in the 70 years since the founding of New China, our country has made revolutionary breakthroughs in various technologies. For example, early nuclear technology based on cybernetics, breakthroughs in supercomputing technology that broke the blockade, and manned space satellite technology have all greatly enhanced the country's strength. After 70 years of industrialization and informatization, we have entered a new era of intelligence. The AI symphonic variation of "Me and My Motherland" by Ping An overcomes technological barriers and, to some extent, brings artificial intelligence technology into a new territory. It will undoubtedly leave a significant mark on the great historical moment."

RECENT AI NEWS

RECENT AI TOOLS