- Video-based searches are coming to Google and it will allow users to record and share a video of their issues and queries.
- This feature will use Gemini to break down the video, understand the context, detect motion, and pick up auditory queries.
- The feature will be available on Google Lens and should roll out to users signed up on Search Labs in the coming weeks.
Google I/O 2024 had a bunch of exciting news surrounding Google’s achievement in the space of generative AI, and multi-modalities. Among the features coming to Google’s suite of apps is a Search with video tool that will change the way you use Google Lens once and for all.
The new Search with video feature inside Google Lens allows you to record videos of anything you’re curious about and share them with Google. It will then use Gemini to find the context of the video and give you the answers you’re looking for.
Senior Director of Google Product Management, Rose Yao did a small live demonstration of how this feature. She recorded a video of a vinyl record player and asked why its tonearm is drifting without actually using the word “tonearm”. Gemini, however, picked up that she wanted to fix the tonearm and provided results accordingly.
The custom Gemini model was able to pick up her voice from the uploaded video. It broke down the video frame-by-frame to learn the device’s make and model and detect motion to understand the issue. This feature going to be available within Google Lens and you should be able to try it out in the coming weeks. Users will have to opt in on Google Search Labs.
This will make searching the web for problems a lot more accessible. Especially for users who have hand impairments, our grandparents who have a tough time navigating through Google searches. What do you think about this feature and are you excited to try it out? Let us know in the comments below.
