I saw a demo of Gemini Live’s multimodal video streaming and screen sharing support at Google I/O 2024, and admittedly, I was skeptical Project Astra would scale down to smartphones well. Fast-forward about a year later, and Project Astra is in the hands of millions of Google Pixel 9, Samsung Galaxy S25, and Gemini Advanced users. It’s ready, and it’s here — but is it actually useful?
The answer is that it absolutely can be, as long as its use cases are personal to you. For instance, Gemini Live’s multimodal features aren’t going to be a better alternative to Google Search or Circle to Search in most cases. However, in specific circumstances where it’s better to show than tell, Gemini Live’s camera support can come in handy.
After weeks of testing Gemini Live with video streaming, the feature has become an irreplaceable part of one of my non-tech hobbies: gardening. I’ve done it all my life, but after moving from the Garden State to the Arizona desert, I had to relearn everything — new growing seasons, care techniques, problems, and everything in between.
To my surprise, it wasn’t Google or Reddit that helped me navigate this climate most. It was Gemini Live.
Gemini can probably identify plants way better than you can
AI can be fun to play around with, but it’s only truly useful if you know when to use it. Specifically, Gemini Live’s camera features offer the most utility when you have something right in front of you that you’re not sure how to describe. Maybe it’s a piece of furniture in a coffee shop you don’t know the name of, or a monument with a faded plaque. In these situations, typing a written prompt into a chatbot or a search engine won’t do you any good.
That’s where you’d want to turn on your camera in Gemini Live. It’ll help out in many situations, including identifying plants. I picked up a few small potted plants from Ikea and turned to Gemini Live to figure out exactly what they were and how to care for them. Here’s how Gemini responded, based on pointing my camera at the plant:
That looks like an aloe vera plant! Is there anything you’d like to know about aloe vera care?
Gemini Live
While this is all happening off-device, Gemini returned the answer pretty fast. Sure enough, it looks like an aloe vera plant. After that, I took the AI assistant up on its offer to enlighten me on care options. Here’s what it said:
Sure, aloe vera plants are easy to care for. They need bright, sunny spots and well-drained soil. You should let the soil dry out between watering. During the spring and summer, you’ll want to water it about every two weeks. In the fall and winter, you can cut back to about once a month.
Gemini Live
The responses were clear and concise, and they ended up saving me a ton of time researching care instructions for a plant I wasn’t even sure the name of a few minutes earlier.
It’ll come up with care options for your plants, too
If Gemini Live were only able to identify plants by looking at them, it would be cool, but it wouldn’t be groundbreaking. The killer feature here is the care instructions that Gemini can provide just by looking at a plant. To test this, I showed it a live camera feed of my basil plant and asked how I should take care of it, and Gemini replied with the following:
That’s a nice basil plant! It needs about six hours of sunlight each day, so a sunny spot is perfect. You should water it when the top inch of soil feels dry, and make sure the pot has good drainage. Do you have a sunny spot in mind for it?
Gemini Live
I then told Gemini I was planning to keep the basil out on my balcony, and it seemed to like the idea.
A balcony is a great spot! Basil thrives with six to eight hours of direct sunlight each day. Just make sure to bring it inside if the temperature drops below 50 degrees or rises above 90 degrees Fahrenheit.
Gemini Live
The cool thing about going “live” with Gemini is that you can easily provide context or follow-up questions in real time. For example, when the AI suggested I bring it inside when the temperature drops, I could let Gemini know that it won’t be an issue in my climate. It’s easy to get context-specific advice when using your camera with Gemini Live.
When issues arise, Gemini Live can diagnose them
By far, Gemini Live’s most useful capability is diagnosing problems you might run into with your plants. As I mentioned earlier, I’m new to growing plants in the desert, and it’s a learning curve. Since the sun is so intense and the temperature can exceed 120 degrees Fahrenheit, the climate is unforgiving. I was bound to run into issues, and Gemini Live helped me navigate them.
In one case, my hydroponic jalapeño and tomato plants were having a hard time. The jalapeño leaves were browning, and the tomatoes just weren’t turning red. It was Gemini that suggested my apartment’s temperature could be the problem, and sure enough, after turning up the heat for a few days, the tomatoes ripened. It also identified health problems with my plants, as explained in the reply below:
Your jalapeño plant looks like it might have a few issues. The leaves are curling and have some brown spots, which could be from overwatering or a nutrient deficiency. The tomatoes also look like they might have blossom end rot, which is caused by a calcium deficiency. Do you have any plant fertilizer on hand?
Gemini Live
It’s important to remember that while you’re reading the transcript of my conversation with Gemini, this is all being spoken to me in real-time — the same way you’d have a conversation with another person.
I realized Gemini Live’s multimodal features were the real deal when it noticed that I was using an AeroGarden hydroponic system just from the information it could see via my camera feed. After picking up on that information, I received tailored care instructions as follows:
Okay, great. You can try adding a balanced liquid fertilizer to the water in your AeroGarden, following the instructions on the label. That should help with any nutrient issues. For the blossom end rot on the tomatoes, you can try adding a calcium supplement to the water.
Gemini Live
While I might usually dive into gardening books, research studies, or Reddit threads for hours to solve these types of problems, everything you see in this article happened within just a few minutes.
Often, plant growing problems can show the same symptoms, making them hard to diagnose, especially when you’re just getting started. Gemini was vital in helping me overcome them.
My plants are stronger than ever thanks to Gemini Live
If you’ve gotten this far and aren’t as excited by Gemini’s gardening prowess as I am, fear not. This is just one way Gemini Live can help you out in your daily life, and there are many more. I used Gemini to solve problems with my plants, but in the tech world, we simply call that troubleshooting.
You could follow the exact same steps for just about anything else you need to figure out. In Google’s first-ever demo of Project Astra at Google I/O 2024, it used the technology to troubleshoot a finicky record player. These are exactly the kinds of things AI can be great at — it is trained on material that includes user manuals, textbooks, and even online forums. You can tap into the online history of the internet without doing any of the work.
So, before you write off Gemini Live, give its new multimodal camera support a try. When you find your use case for it, it’ll become invaluable.
The best Google smartphone
I used a Google Pixel 9 Pro XL to help my plants with multimodal Gemini Live, and you can too. It’s free with the Pixel 9 series, and you get a bunch of other Google AI features as well. Plus, there’s a big screen, impressive camera system, and Tensor G4 processor.
