- Day 6 of 12 Days of OpenAI Brought Visual Capabilities to ChatGPT Advanced Voice Mode
- You can stream video from your camera or share your screen with AI
- Only ChatGPT Plus and Pro subscribers have access for now
ChatGPT looked at the sixth day of the 12 days of OpenAI with a fresh look, courtesy of a new visual capability connected to its advanced voice mode that allows you to share your screen and stream live video to the AI chatbot. Instead of uploading photos and screenshots to ask questions to ChatGPT, you can now show it what you're looking at directly and ask for advice.
The feature is a bit like having a video chat with a friend, although you don't share your own image. But you can hear the AI voice, which makes the conversation hands-free. If you want to try it, you can tap the voice icon in the ChatGPT app and then the video icon to start streaming video from your camera. To share your screen, simply tap the three-dot menu and select “Share Screen.”
Imagine you're struggling to put together a new IKEA shelving unit (because who hasn't done it?). Instead of staring at confusing instructions, you can point your camera at the half-built mess and ask ChatGPT, “What did I do wrong here?” AI can check parts and offer step-by-step guidance.
If you need to change some settings on your phone or computer and aren't sure how to do it, you can share your screen with ChatGPT to let the AI guide you through menus and buttons to sort things out. No more searching through endless tech forums or asking your friend who's good with computers.
The feature could even make ChatGPT a better kitchen helper. If your recipe says “whisk until thick” and you're not sure you've blended enough, simply point the camera at your bowl and ask ChatGPT for confirmation or a suggestion about whether you might want to order.
All-seeing AI eyes
OpenAI CPO Kevin Weil and his team demonstrated how ChatGPT can help brew coffee by pointing the camera at your brewing setup to show off the new feature during its debut. The AI understood the coffee-making equipment and guided them through the steps like a virtual barista.
The feature encourages ChatGPT to be treated more like a person than a computer, even more so than just the voice interface. Being able to “see” makes AI seem more present in the real world and less like a chatbot in a vacuum.
OpenAI acknowledged that sharing what your camera sees could make some wary of using the feature. It does not activate automatically, you must activate it every time you use it so that a video is not accidentally recorded.
The new feature is only available for ChatGPT Plus and Pro users as of now. Enterprise and Education tier subscribers will have access next month, but OpenAI did not say if or when the free tier will have access.
That makes sense since it probably demands a lot of computing power and OpenAI doesn't want ChatGPT to go down again after what happened on Wednesday.
