Artificial intelligence for video production has become increasingly popular thanks to tools like Sora, Runway and Pika. But a new AI-augmented cinema camera has taken the idea to the next level.
Creative technology agency SpecialGuestX and mixed-media production company 1stAveMachine have introduced the CMR-M1 to address that goal. The result is a camera that doesn’t just film the world around it, but enhances it using a generative AI-powered video-to-video model. The digital film is sent to the cloud, where a stable diffusion-based model transforms it. What’s more, the AI can also link with other APIs to further play around with what you can do with the film.
“We believe that filmmaking is inherently physical. Filmmakers are used to using cameras, lenses, tripods, lighting, and that AI has to be a new tool that enhances creativity,” said Miguel Espada, co-founder and executive creative technologist at SpecialGuestX. “In the spirit of creative experimentation, we designed a camera that serves as a physical interface for AI models.”
The camera features a wheel on the side of the housing, allowing operators to seamlessly switch from real overlays to AI-generated overlays. There’s even a slot for a style card containing an NFC chip that connects to the user’s model trained with their custom images and cues.
While this isn’t a real-time setup due to latency, developing models like StreamDiffusion could make this a reality. Currently, there are five different filters the film runs through, including the jungle-like “Blooming Nature” and the tuxedos and gold of “Old Money.” You can see an example from “Old Money” below.
AI Movie
The CMR-M1 is currently a prototype and is not intended for commercial release. It is part of SpecialGuestX’s research into creating physical interfaces for generative AI. However, the design of the CMR-M1 has been developed with scalability in mind, allowing for possible commercial production in the future.
The CMR-M1, which has a rather basic name, Camera Model 1, deliberately attempts to evoke the film origins in its appearance. The camera has a design reminiscent of the 1924 Cine-Kodak camera, one of the first and most popular portable 16mm cameras.
“Sometimes, to imagine what the future could be like, you have to create a prototype. This is especially true when it comes to film production, creative technologies, and the creative community,” said Aaron Duffy, Founder and Executive Creative Director of SpecialGuest. “The CMR-M1 is a prototype for how hands-on creators will use AI instead of sitting at a keyboard.”
It’s hard to say whether the CMR M1 will have an impact on the wider filmmaking world. The more immediate question is whether it will open up a new avenue of competition between AI video model developers such as Runway and its new Gen-3 Alpha, OpenAI and its Sora model, and efforts from Pika and Luma Labs.
The inclusion of open-source models from Stability AI suggests that such access may play a big role in the evolution of AI videos, though what appears in the final reel of the competition is still largely a cliffhanger.
