Subway Surfers, one of several games being reimagined by AI music innovators XDMind.
Machines are getting smarter — which means the music industry is taking AI and the metaverse more seriously. According to the duo behind XDMind, gaming is a perfect place to unleash AI-powered adaptive music technology.
A game without music is like a car without a stereo system. No fun. But what about all the gamers that mute their games while listening to their own Spotify playlists? Despite the presence of brilliant soundtracks and fantastic musical integrations in games like Grand Theft Auto, there seems to be plenty of room for evolution in this arena.
Now, startup XdMind is trying to fill the ‘mute gap.’ Using adaptive AI technology, XdMind’s GameDJ allows a gamer to incorporate any preferred music into any game. But instead of a parallel stream, GameDJ smartly integrates music directly into the action. That means intensifying the music around more action-heavy sequences, for example, and toning things down during downtimes (including non-action modes like character selection). The result is a more responsive, action-matching musical soundtrack powered by AI.
“A large percentage of gamers like to listen to their own music,” relayed XDMind cofounder Ara Bernardi, a 20-year Microsoft engineering veteran. “I’m a case study in this: I exclusively listen to my music while gaming, especially when the in-game music gets boring or irritating.”
“There are so many gaming soundtracks on Spotify for a simple reason: gamers want more than in-game music.”
XdMind’s “your games, your music” concept has already attracted serious interest from the gaming industry and its investors (currently, XDMind is privately funded).
According to XDMind, gaming developers have struggled with musically-disconnected gamers for years, with mixed success. But the prospect of an integrated, adaptive music soundtrack offers the possibility of greater musical choice and personalization — and as a result, significantly increased engagement.
“Game makers don’t really know what music you like; only you do,” said XdMind fellow cofounder Linda Bernardi, a seasoned tech disruptor and innovator. “Music is highly personal, and games can be enjoyed more dramatically if your favorite music is adapted to your favorite game, seamlessly and without any interruption to your gameplay.”
“We enable true music personalization and dynamic adaptation of music to video games via AI as never done before, achieving true personalization.”
The first iteration, viewed by Digital Music News, offers a nice preview of the possibilities (DMN became a pre-launch partner of XDMind late last year). The Android-specific concept uses personal collections on mobile games like Subway Surfers and Candy Crush. The first step is to swap the in-game music with a track selected by the user. From there, the action of the music is paired with the action of the game.
GameDJ is designed to integrate immediately, without extensive setup or app updates.
Once engaged, GameDJ works alongside the game without forcing any interruptions or distractions. According to XDMind, that ‘side-by-side,’ out-of-the-box development philosophy will improve early engagement beyond supergeek gamers. The company is also resolutely avoiding ads, tracking, or other pesky intrusions.
Over time, GameDJ adapts to the game to improve the fit using its video comprehension technology. Eventually, the goal is to create a contextually harmonized experience that properly matches certain sections of songs — while also subtly modifying those songs — to fit the action and enhance the in-game experience.
The first version of GameDJ is an excellent step toward that vision. Initially, GameDJ is relying on fade-ins and fade-outs to transition its matches, though eventually, more advanced techniques like looping, stylistic modifications, and even mashups will be employed in future versions, according to the company.
That is fresh stuff for the gaming industry, but GameDJ is also drawing interest from the music industry.
An ‘opportunity’ snapshot from XDMind’s pitch deck.
Both industries are enjoying breakneck growth and untapped expansion opportunities. Accenture recently valued the global gaming industry at more than $300 billion, with mobile gaming spawning hundreds of millions of new gamers over the past few years alone. That is happening alongside a massive app boom: according to App Annie’s latest data, a record 230 billion apps were downloaded in 2021, with total app spending surging 19% in 2021 to $170 billion.
The music industry is also enjoying a spectacular surge: according to data released by global industry trade group IFPI, worldwide recording revenue jumped 18.5% to $25.9 billion in 2021. But revenue from streaming platforms like Spotify, which have been powering the surge, is starting to cool. Suddenly, the industry is scoping out new platforms for growth, and gaming and the metaverse are topping the list.
In its current iteration, GameDJ only uses locally-stored, user-owned music collections and marries those songs with an existing game.
That structure is designed to flex the proof-of-concept without triggering any copyright concerns. The next step is to introduce greater musical options and license catalogs.
On that front, XDMind is already in advanced discussions with major music IP owners. The company noted that the discussions involve “very large libraries of music for streaming,” with considerable flexibility in usage parameters and creative modifications.
At first, the prospect of AI-bending and matching might sound scary to major content owners. But within the confines of a gaming add-on, the calculations change quickly. “GameDJ allows gamers to extend their music in totally different ways and within a controlled environment,” Linda relayed. “I think they see the upside pretty quickly.”