Google is reportedly working on rolling out several customization features to the Live Caption accessibility feature on mobile devices. Evidence of this update was discovered by Assemble Debug software after digging into the Android System Intelligence app. According to an image provided to Android Authority, there will be four options in total. We don't know much, but there is a little explanation to be found.
The first allows Android phones to display “emoji icons” in a caption transcript; perhaps to better convey what emotions the voices express. The other three are not so clear. The second characteristic “will emphasize the emotional intensity in [the] transcription”, while the third is said to include the “duration of the word [effects]”and the ability to display “emotional labels.”
Feature Breakdown
As you can see, the wording is quite vague, but it's enough to give you an idea. It seems that Live Caption will get better at replicating the emotions in the voices it transcribes. Say, for example, you're watching a movie and someone shouts angrily. Live Caption could perhaps display text in all caps to indicate that you are shouting.
The feature could also tilt words in a line to indicate when someone is being sarcastic or trying to imply something. The word duration effect could refer to the software displaying drawn letters in a set of subtitles. Maybe someone is singing and starts holding a note. The sound that is being held can be displayed thanks to this switch.
It is true that emotional labels are more difficult to imagine. Android Authority mentions that tags will be displayed and included in a transcript. This could mean that the tool will add clear indicators within the transcripts of what a subject is expressing at that moment. Users can see the word “Angry” appear whenever a person feels angry about something or “Sad” appear whenever someone cries.
Greater utility
That's our best guess. If these rumored features work as described, it would give Live Caption even more utility than it already has. The tool was introduced in 2019 as an accessibility tool to help people enjoy content if they are hard of hearing or unable to unmute for any reason.
The current subtitles are pretty simple, but with the update, emotions could be added to Google's tool for a better immersive experience.
Android Authority claims that the features were found in a “variant of the Android System Intelligence app.” We believe this means they were located within a special version of the app intended for proprietary hardware like the Google Pixel. So the customization tools may be exclusive to the Pixel 8 or a future model. It's too early to tell at this point. Hopefully the updated Live Captions will see a much wider release.
Until we know more, check out TechRadar's list of the best Android phones for 2024.
