Almost everyone in tech is investing heavily in artificial intelligence right now, and Google is among the most committed to an AI future. Project Astra, unveiled at Google I/O 2024, is a big part of that and could end up being one of Google's most important AI tools.
Astra is presented as “a universal AI agent that is useful in everyday life.” It's essentially something like a combination of Google Assistant and Google Gemini, with added features and boosted capabilities for a natural conversation experience.
Here we'll explain everything you need to know about Project Astra: how it works, what it can do, when you'll be able to get it, and how it could shape the future.
What is Project Astra?
In some ways, Project Astra is no different from the AI chatbots we already have: you ask a question about what's in an image, or how to do something, or request that some creative text be generated, and Astra gets ahead with it. that.
What elevates this particular AI project is its multimodal functionality (the way text, images, video and audio can be combined), the speed at which the robot works, and its conversational level. Google's goal, as we've already mentioned, is to create “a universal AI agent” that can do anything and understand everything.
Think of the Hal 9000 robot in 2001: A Kubrick Space Odysseyor Samantha's assistant in the movie. His: Talking to them is like talking to a human being and there's not much they can't do. (Both AIs eventually became too large for their creators to control, but let's ignore that for now.)
Project Astra was created to understand the context and take action, to be able to work in real time and remember conversations from the past. Based on the demos we've seen so far, it works on phones and smart glasses, and it works with Google Gemini AI models, so it may eventually be part of the Gemini app, rather than something separate and independent.
When is Project Astra coming out?
Project Astra is in its early stages: this is not something that will be available to the masses for at least a few months. That said, Google says that “some of these agent capabilities will come to Google products like the Gemini app later this year,” so it looks like elements of Astra will gradually appear in Google apps as we move forward. until 2024.
When we had hands-on time with Project Astra at I/O 2024, these sessions were limited to four minutes each, which gives you an idea of how far this is from being something anyone, anywhere, can use. . What's more, the Astra kit didn't seem particularly portable, and Google representatives were careful to refer to it as a prototype.
Taking all that together, we have a feeling that some of the Project Astra tricks we've seen in a demo could appear in the Google Gemini app sooner rather than later. At the same time, the full Astra experience, perhaps with some dedicated hardware, probably won't roll out until 2025 at the earliest.
Now that Google has shared what Project Astra is and what it's capable of, we'll likely hear a lot more about it in the coming months. Keep in mind that OpenAI, developer of ChatGPT and Dall-E, is busy releasing its own major updates, and Google won't want to be left behind.
What can I do with Project Astra?
One of Google's demos shows Astra running on a phone, using its camera input and speaking naturally to a user: it is asked to point to something in sight that can play sounds and correctly identifies a speaker. When an arrow is drawn on the screen, Astra recognizes and speaks to the speaker component highlighted by the arrow.
In another demonstration, we see Astra correctly identifying world landmarks from drawings in a sketchbook. She is also able to remember the order of objects in a list, identify a neighborhood from an image, understand the purpose of sections of code shown to her, and solve mathematical word problems.
A lot of emphasis is placed on recognizing objects, drawings, text and more through a camera system, while also understanding human speech and generating appropriate responses. This is the multimodal part of Project Astra in action, making it a step up from what we already have, with improvements to caching, recording and key processing for real-time responsiveness.
In our hands-on time with Project Astra, we were able to get it to tell a story based on objects we showed to the camera and adapt the story as we went. Later, it's not hard to imagine Astra applying this intelligence while exploring a vacation town, solving a physics problem on a whiteboard, or providing detailed information about what's being shown in a sports game.
What devices will Project Astra include?
In the Project Astra demos that Google has shown so far, the AI runs on an unidentified smartphone and an unidentified pair of smart glasses, suggesting we may not have heard the last of Google Glass yet.
Google has also hinted that Project Astra will come to devices with other form factors. We have already mentioned the His movie, and it's entirely possible that we'll eventually see the Astra robot integrated into wireless headphones (assuming they have a strong enough Wi-Fi connection).
In the practical area that was installed at Google I/O 2024, Astra worked through a large camera and could only work with a specific set of objects as props. Clearly, any device running Astra's impressive features will need a lot of onboard processing power or a very fast connection to the cloud to maintain the real-time conversation that is critical to AI.
However, as time goes on and technology improves, these limitations should slowly begin to be overcome. The next time we hear something big about Project Astra could be around the time of the Google Pixel 9 launch in the final months of 2024; Google will no doubt want to make this the most AI-enabled smartphone yet.
