An 8-bit plumber with a mustache and a taste for mushrooms was the height of cutting-edge entertainment in 1985. Now, an AI model can recreate the entire game based on a few basic cues (and sometimes hilariously fail in the process).
The MarioVGG demonstrated in a new academic paper by video game character developer Virtuals Protocol shows how AI could help collaborate in future AI-powered video game production—or possibly why it shouldn't.
MarioVGG is an experiment in getting AI to produce plausible gameplay videos from a set of cues about how the environment should look and behave and how characters should act. It's a long-form version of the kind of text-to-video tools that are rapidly gaining popularity. It's trained on over 737,000 frames of Super Mario Bros. gameplay footage.
Mario's AI doesn't do everything the character in the Nintendo game does, though, as the researchers limited the model's interaction capabilities to just two moves: “run right” and “run right and jump.” The resulting footage forms a video, though one suggests the player deliberately made it more challenging by avoiding power-ups, only jumping to a height, or moving to the left.
Below, you can see a bit of MarioVGG's performance. It's surprisingly good at recreating the look of the game, but it's also riddled with glaring bugs and flaws. While the AI-generated sequences are mostly consistent at matching user input to how Mario acts, it's nowhere near as fast, with the AI taking a while to create each frame. Not to mention Mario occasionally disappearing from the screen or transforming into an enemy character for a moment.
Super MarAIo Modeling
Regardless, seeing AI display a partial understanding of cause and effect between user input and the resulting game brings no-code game development that much closer to reality. AI models like MarioVGG aren’t going to replace game developers or the standard engine anytime soon, but the idea of simply explaining to an AI how you want a game’s physics and environments to work instead of coding everything by hand is too tempting to be deterred by technical hurdles.
There is still a lot to be solved, as video games involve a series of constantly evolving interactions between the player and the game environment. That is much more complex than static images or simple actions in a video. Accurately recreating this real-time interactivity is a challenge that MarioVGG has not yet solved, although MarioVGG is not the only project in that direction.
Google’s GameNGen recently showed off a playable version of Doom produced by an AI model. Despite being an older game, Super Mario Bros. requires more nuanced control over character movement and interaction with the environment than Doom does, so the result isn’t as fast or accurate. Currently, AI is much better at text-based games with occasional illustrations than it is at something like a Mario game. But, like the early days of video games, an evolution toward the fast-paced, interactive video games of Nintendo’s golden age could come faster than any Goomba could.
