Android’s new feature bundle brings captions that actually show emotions
Key Takeaways
- Google’s December Android Feature Bundle focuses on AI, accessibility, and user convenience.
- Expressive Captions adds tone and environmental cues for live captions on Android 14 and above, while Google Drive now optimizes scanned documents with enhanced contrast and white balance.
- Google’s Lookout – Assisted vision app is now powered by Gemini 1.5 Pro and users now have the option to use QR codes to share files via Quick Share.
Google today announced a fresh batch of features arriving on some of the best Android smartphones as part of its December 2024 Feature Bundle, with a core focus on AI, accessibility, and user convenience.
No Thanks, Keep Reading
The bundle follows the September 2024 drop, which brought features like TalkBack image descriptions, song identification in Circle to Search, web page read-aloud playback on Chrome, the expansion of Earthquake Alerts, and more.
Google’s latest Android feature bundle brings offline maps to Wear OS and Earthquake Alerts to the entire US
TalkBack also gets a vision upgrade
The highlight of the feature bundle comes as a solid upgrade to Android’s Live Caption accessibility feature. Back in August, Google expanded Live Caption support to seven new languages — namely Korean, Polish, Portuguese, Russian, Chinese, Turkish, and Vietnamese. Today, the tech giant is making these captions a lot more ‘expressive.’
Aptly named Expressive Captions, the new feature “will not only tell you what someone says, but how they say it.” Not to be confused with Live Transcribe, the new feature essentially offers captions for sound coming from your device — not the sound in your surroundings.
Google suggests that Expressive Captions are powered by “multiple AI models,” and can communicate tone, volume, environmental cues, and even human noises. All Caps captions would reflect the intensity of speech, while sighing, grunting, gasping, and similar sounds will clearly be labeled. Further, “We’ll label additional noises in the foreground and background, like applause and cheers, to give you a fuller picture of what’s happening in the environment,” wrote the tech giant.
Expressive Captions are available across the US starting today on devices that support Live Captions on Android 14 and above.
As part of the bundle, Google’s Lookout – Assisted vision app is also receiving a major upgrade. Back in 2022, Google made it so that the app could function without an internet connection. The feature, which essentially uses AI to generate image descriptions for the content on your screen, alongside the ability to answer follow-up questions, is now supercharged with Gemini 1.5 Pro. This should result in more descriptive and richer image descriptions when compared to previous iterations.
Simply take, upload or open a photo in the app to hear the caption read aloud in a natural-sounding voice. You can then gain a deeper understanding of the image by asking follow-up questions, now available globally.
Scan and share without hassle
Elsewhere, users now have an easier way to transfer pictures, videos and documents to and from their Android devices. As highlighted by Google, users now have the option to use QR codes in Quick Share, eliminating the need for contact additions or device verification. Hints about QR code functionality first started emerging with Google Play Services version 24.20.13, all the way back in May.
The last major feature pertains to Google Drive enhancements. Forms, receipts, ID cards, or other documents that you scan via your device will now automatically be optimized with improved contrast and white balance, paired with the removal of shadows and blurred elements — all without the need to manually edit anything.
To use the new feature, make sure that you scan documents via the camera icon in the Google Drive app. Post-scanning, you should be able to apply enhancements to the document by tapping the “Enhance” button on the preview screen (the one with the Gemini logo). You then have the option to name the file and choose whether it’s saved as a PDF or a JPEG.
Lastly, and this isn’t really news-news — Google today made several Gemini extensions, ones that we’ve seen rolling out in the past month, official. This includes the Spotify extension that began rolling out late in November, extensions for Phone, Messages, and Whatsapp that began rolling out yesterday, and the Utilities extension that rolled out earlier this week. The tech giant also highlighted that users will soon be able to use Gemini to control their smart home devices, suggesting that the Google Home extension might soon leave Public Preview.
Google Home extension for Gemini rolling out widely in Public Preview
Say goodbye to complex commands
