“The Hardest Part,” a new song by indie pop artist Washed Out, is about lost love, one of the most human topics.
But ironically, to illustrate the melody's sense of longing, the musician turned to something much less flesh and blood: artificial intelligence.
With Thursday's release of “The Hardest Part,” Macon, Georgia-based Washed Out, whose real name is Ernest Greene, has the first collaboration between a major music artist and a filmmaker on a music video using technology from text to video Sora from OpenAI. , according to the singer-songwriter's record label, Sub Pop.
The approximately four-minute video, directed by Paul Trillo, quickly takes the viewer through key elements of a couple's life. The audience sees the characters, a red-haired woman and a dark-haired man, go from kissing and smoking in a 1980s high school to getting married and having a child. “Don't cry, everything's okay now,” Greene sings. “The hardest part is that you can't go back.”
The couple is not played by real actors. They are created entirely digitally through Sora's AI.
The video could usher in a potentially groundbreaking trend of using AI in video production.
“I think the place we're in right now is about to explode, so I'm looking forward to incorporating some of this new technology and seeing how that influences what I come up with,” Greene said in an interview. “So if that's groundbreaking, I'd love to be a part of it.”
“The Hardest Part,” the lead single from Greene’s new self-produced album “Notes From a Quiet Life,” scheduled for release June 28, is the longest music video made with Sora technology to date. The program creates short clips based on written text prompts. This allowed Trillo to construct scenes in a way that would have been much more expensive with real actors, sets, and locations.
“By not having budget constraints or having to travel to different places, I was able to explore all of these different, alternative outcomes of this couple's life,” Trillo said.
Trillo is one of the creatives who has early access to Sora, which is not yet publicly available. OpenAI introduced Sora in February and has been testing the system with directors and meeting with Hollywood executives and producers. He is troubleshooting and trying to address intellectual property concerns.
Innovations in AI have been hugely controversial in many corners, including the music industry, which has been plagued by the use of “deepfakes,” or videos and audio that falsely use an artist's image or voice. Musicians and others have pushed for legislation to combat such deceptive creations, and talent agencies are working with tech startups to clamp down on unauthorized digital imitation.
The introduction of Sora, coming from the same company that created the text-based AI model ChatGPT, raised concerns within Hollywood and elsewhere about its potentially devastating impact on jobs and production. Still, it inspired excitement among some creatives for the ways it could help them achieve their vision on screen without being limited by special effects budgets and travel limitations.
Both Greene and Trillo said they were able to do more with Sora than they would have done with real-life sets within their budget. Sub Pop did not disclose the costs of the video. The music artist did not pay OpenAI to use the technology in the music video.
The two men had explored other ideas, including hiring dancers and filming in a location that resembled the green hills on the artwork for Greene's new album, but this proved difficult due to time and financial constraints. Trillo then suggested experimenting on Sora.
Greene, whose music television audiences may recognize from the theme song to the satirical comedy show “Portlandia,” was hesitant at first.
“I feel like my music and most of the videos I've made over the years always start from a really emotional and sincere place,” Greene said, noting that many of the examples of AI videos he had seen existed. in the dreaded “uncanny valley,” human-like but disturbingly artificial.
However, Greene was willing to experiment. So Trillo tried different concepts to see what would work in the video. Using technology, he was able to explore the various outcomes of the couple's lives in multiple locations by creating elaborate text-based prompts. He completed the video in about six weeks, editing together about 55 clips into the video out of the approximately 700 he generated using Sora.
“With this, I didn't have to edit myself,” Trillo said. “I really got to try things out and that organically creates a different type of story because of that, being able to throw so many things at the wall and see what sticks.”
To generate usable clips, Greene needed to write prompts with enough specific details about not only the image itself but also the shooting angles and character movements. “We go through the bursting bubble, through the gum, and into an open soccer field,” Trillo wrote as part of his suggestion for a short video clip. “The scene moves quickly, showing a frontal perspective, showing the students getting bigger and faster.”
The final music video for “The Hardest Part” shows various locations, including a high school, a grocery store, hills, a hallway with white sheets billowing, and fire burning through the walls.
Ernest Greene, known as the “Washed Out” musical artist, will release a new album on June 28.
(Ernest Green)
There were some limitations. Sometimes Trillo would have an idea and Sora would make it happen. Other times, he would create something chaotic and unusable. The videos appeared with inconsistencies, which Trillo sometimes preferred to overlook. The characters look a little different from clip to clip, as does the couple's son.
Part of the video's artistic charm is its dreamlike state: memories of a couple's life that illustrate the darkness of human memory.
“You have to know where to pick your battles,” Trillo said of Sora. “You have to give up a little bit of your free will in working with this and you have to accept the nature of how chaotic it is.”
“I was certainly impressed with how far he could go in piecing together a story,” Greene said.
Both Greene and Trillo said they see AI as potentially opening up more opportunities for people to push the music video art form. Music videos are a logical medium to play with AI, because they are generally short and cost much less than movies and TV episodes.
However, Trillo said, it's important to him that this not be used as a new primary method of creation, but rather as another tool in the tool belt.
“A lot of music videos just don't have the budget to dream big,” Trillo said. “I think AI can help the music industry in terms of creating things that even Ernest could dream of and maybe wouldn't have dared to dream of before.”
