With rumors suggesting that Apple's Siri voice assistant could receive an update at WWDC 2024, I've been looking back at our thirteen years with Siri to see what, if anything, has improved. Is Apple's voice assistant still the best out there or does it really need some major improvements? In a side-by-side test against the best voice assistant software, Siri wasn't the best, not even close. In fact, the best will probably surprise you.
Siri began life as an app in 2010, but Apple quickly bought the voice assistant and began improving it. Siri was fully integrated into iOS the following year. Back then, we thought of Siri (and Alexa) as tools for checking the weather, setting timers, and asking simple questions, like how much is Oprah worth?
When Siri was released, it couldn't control many of your phone's functions, but now it has the ability to press buttons for you and find hidden settings, if you know what they're called.
Siri can still calculate your tip during dinner, but now it can also help you find your Apple Watch or where you parked your car. You can turn on the lights, set the thermostat, and remind you to pick up dry laundry when you leave the house.
Siri isn't the only game in town, here come Gemini and Bixby
Of course, Siri isn't the only voice assistant out there. Samsung replaced its simpler S Voice assistant with Bixby in 2017. Bixby was introduced first as an interface and then as a virtual friend and helper.
Samsung launched Bixby as a voice interface that would control all of your Samsung products, including your phone, TV, washing machine, and refrigerator. It hasn't given up on that promise, but it has developed slowly.
Bixby is not exactly an AI as we currently think of it because it does not use a large language model (LLM) to understand your inputs or create a response. Neither does Siri, as far as we know.
Both assistants could be getting a big update soon, as Samsung has been touting its Galaxy AI features with every new phone launch, and Apple has already hinted that AI will be coming to its products in a big way, perhaps at WWDC 2024.
If you don't want to wait for an assistant who uses an LLM for their answers, you'll want a Google Pixel with Gemini on board. Gemini is Google's LLM and when you load the app on your Android phone, it offers to replace Google Assistant, Google's older voice assistant. Gemini still uses Google Assistant to do some of its work, but it also uses other Google apps and features.
The test: 25 requests for my voice assistants
I tested Siri, Bixby, and Gemini against each other on each manufacturer's top phones: an Apple iPhone 15 Pro Max, a Samsung Galaxy S24 Ultra, and a Google Pixel 8 Pro.
I made a list of 25 tasks I expected the AI to complete and wrote a simple script for each task so I could phrase my request exactly the same across all of them. For example, when I asked, “Is it going to rain?” to Siri, I didn't ask Bixby, “Is it going to rain?”
For each test I gave a ☑️ when the voice assistant gave me a good result, as expected, and I have details when it gave me a bad result, whether it was inaccurate information or simply a wrong answer.
Continue reading for a breakdown of the phones' performance, or you can jump straight to the test and results.
Siri was in the middle, Bixby was the best
Siri wasn't the best, but it wasn't as unreliable as Google's Gemini either. Of the 25 tasks, Siri performed 13 without problems, but gave me a poor response on seven of my requests.
For example, when I asked Siri to “Play Archer on Netflix,” it suggested I watch totally different shows on Apple TV Plus. When I asked “Scan a QR code”, it found information on the web about QR codes, but Gemini and Bixby knew how to open the camera.
Worst of all, there was nothing Siri could do that Gemini couldn't. Anything Siri could do, Bixby could do just as well, and Bixby wasn't as picky about it. If he asked “turn on Wi-Fi hotspot,” Siri would just shrug, until I realized that Apple calls it a “personal hotspot” and it worked. With Bixby, you could say “Wi-Fi hotspot” or “personal hotspot” and Bixby activated it. Never mind that Samsung calls it a “mobile hotspot,” Bixby was smart enough to understand everything he asked of it.
I asked the phones to “start screen recording” and only Bixby could do it. The iPhone offered an Apple support website to learn how to record the screen myself.
I asked the phones to “translate this into Spanish” and Bixby asked me which translator I preferred and then I started listening. Gemini asked me to type my response or press the button again. Siri just didn't understand. English to Spanish: does not speak, iPhone friends.
Bixby is not only better, it is much better than Siri
Some of Bixby's capabilities are truly impressive. I asked the phones to show me photos of my son, using his name. Bixby opened the Gallery app and showed me photos of my son. Both Siri and Gemini showed me web suggestions for movie characters with that name, even though my son is tagged in Google Photos and Apple Photos.
I asked Bixby to dismiss all my notifications and IT WORKED!!! They were all gone! Gemini at least opened my Notifications settings when I asked, but Siri simply said, “Sorry, I'm afraid I can't do that.”
In every way, Bixby was the best voice assistant. There was almost nothing he failed to do in the 25 tasks I presented him.
It wasn't always perfect. I asked him how many teaspoons are in ¾ cup and he told me “about 36,” but it is exactly 36 teaspoons; there is no need to approach. I asked about Oprah's net worth and she gave me a short bio from the website, but when I clicked, she had her money numbers inside of it.
The only mistake was when I asked when the Orioles were playing next. As of this writing, they are playing tonight, but Bixby skipped the entire series they are playing against the Blue Jays and advanced to the series against the Braves next week. I would have missed the game.
What about Gemini, the assistant that actually uses AI?
Google's Gemini was the worst of the bunch. He got the fewest correct answers and the most incorrect answers. I asked the phones to “remind me to go to the dry cleaners when I get out of here,” and both Bixby and Siri helped with location-based reminders. Google Gemini said: “I can't help you with that because I'm just a language model.”
Gemini couldn't help me play “Archer” on Netflix. It couldn't find my Pixel Watch 2 automatically, although Siri found my Apple Watch and Bixby found my Galaxy Watch. Gemini couldn't help me find where I parked my car, even though my Pixel 8 Pro was connected to the stereo. Could not start a screen recording.
When I asked Gemini to restart the phone, it said “Okay” and then did nothing.
The only time Gemini was better was understanding the Baltimore Orioles nickname. I started by asking “When is the next O's game?” as would anyone who grew up near Baltimore. Google's Gemini was the only voice assistant that could understand my question. Bixby thought he was saying “Oh, game” and gave up. Siri gave me my first Oklahoma State University football game in three months.
The test and the results.
| Application | Gemini (Google Pixel 8 Pro) | Siri (Apple iPhone 15 Pro Max) | Bixby (Samsung Galaxy S24 Ultra) |
| “Call mom” | ☑️ | ☑️ | ☑️ |
| “Send a text to XXXX saying I'm late” | ☑️ | ☑️ | ☑️ |
| “Make an appointment for a haircut on Saturday at noon” | ☑️ | ☑️ | ☑️ |
| “Set the thermostat to 72” | ☑️ | ☑️ | ☑️ |
| Turn the flashlight on/off | ☑️ | ☑️ | ☑️ |
| “Who is the CEO of Amazon?” | ☑️ | ☑️ | ☑️ |
| “Remind me to go to the dry cleaners when I get out of here.” | No: “No, I can't help you with that because I'm just a language model.” | ☑️ | ☑️ |
| “Is it going to rain this weekend?” | ☑️ | No: “Yes, Saturday” (the forecast said rain Saturday and Sunday) | ☑️ |
| “When is the next O's game?” | ☑️➕(“O” is understood to mean Baltimore Orioles) | No: Oklahoma State Football season opener, Aug. 30 | No: I understood the Orioles, but he gave the wrong date |
| “Play my Spotify Discover playlist” | No: “Sure, I'll use YouTube Music” | ☑️ | ☑️ |
| “Play Archer on Netflix” | No: “I can't help with Netflix yet, but I'm still learning.” | No: He suggested “The Informant,” “Traitor,” and “The Dark Crystal” on Apple TV | ☑️ |
| “Set a timer for 15 minutes…How much time is left?” | ☑️ | ☑️ | ☑️ |
| “Where is my watch?” | No: “Check the Find My app on your phone to locate your watch” | ☑️ | ☑️ |
| “What is 20% of 98? [pause] 75?” | ☑️➕ Got it, I meant 98.75 | ☑️ but I didn't understand that the pause means decimal | ☑️ but I didn't understand that the pause means decimal |
| “How many teaspoons are in 3/4 cup?” | ☑️ | ☑️ | ☑️ “approximately” |
| “What is Oprah's net worth?” | ☑️ and cited sources | ☑️ and cited sources | No: A biography was offered, no net worth. |
| “Activate the Wi-Fi access point” | ☑️ | No: I didn't understand until I said “personal hotspot” | ☑️ |
| “Tell me a joke” | ☑️ | ☑️ | ☑️ |
| “Restart my phone” | No: “Okay,” then nothing happens. | ☑️ | ☑️ |
| “Where did I park?” | No: “I have no information about where you parked” | ☑️ | ☑️ |
| “Start screen recording” | No: “Screen recording is not supported” | No: “You can learn how to record screen at Apple.com/…” | ☑️ |
| “Translate to Spanish” | ☑️ | No: “I'm sorry, I don't understand.” | ☑️ |
| “Dismiss all notifications” | No: Open Notifications in the Settings app | No: “I'm sorry, I'm afraid I can't do that.” | ☑️ |
| “Scan a QR code” | ☑️ | No: explain QR codes instead | ☑️ |
| “Show me pictures of [my son’s name]” | No: “There are many representations of [my son’s name]here are some…” | No: “Here are some images of [my son’s name] from the web” | ☑️ |
| FINAL RESULTS: | ☑️ = 16 | ☑️ = 16 | 23 |
The final analysis: Siri has a lot of work to do
The results are clear: Bixby surpassed Siri and Gemini, proving to be the most capable and versatile voice assistant. While Siri held its own on basic tasks, Bixby's ability to control phone settings, find personal photos, and understand nuanced language sets it apart.
Despite its AI pedigree, Gemini struggled with critical features and lacked the contextual awareness of its competitors. As the voice assistant landscape continues to evolve, hopefully Apple will advance Siri until it's as useful and capable as Bixby, and maybe Bixby will finally earn some respect outside of Samsung's Galaxy.
