Of the four major smart speakers launched by Amazon, Google, Apple and Microsoft, Google Assistant is the smartest voice assistant, followed by Apple Siri, according to the Annual Smart Speaker IQ Test conducted by Loup Ventures, a US-based venture capital firm investing in frontier technologies.
Of the four major smart speakers launched by Amazon, Google, Apple and Microsoft, Google Assistant is the smartest voice assistant, followed by Apple Siri, according to the Annual Smart Speaker IQ Test conducted by Loup Ventures, a US-based venture capital firm investing in frontier technologies.
Tests were done on the 2nd-generation Amazon Echo, Google Home Mini, Apple HomePod and Harman Kardon Invoke. Google appears to be the smartest. The result shows that Google Assistant accurately answers 88% of the questions, followed by Siri at 75%, Alexa at 73% and Cortana at 63%.
Google Assistant also got a 100% score on the ability to understand test questions. Other voice assistants, on the other hand, are also near perfect in this regard with their scores all hovering above 99%.
“Both the voice recognition and natural language processing of digital assistants across the board has improved to the point where, within reason, they will understand everything you say to them,” Loup Ventures says.
Loup Ventures conducted the test periodically and 800 questions spanning five categories were asked: “local” questions gauge the voice assistant’s ability to retrieve information like local shops and restaurants; “commerce” questions gauge their ability to aid users purchase merchandise; “navigation” questions test their ability to provide the right direction to help a person move from point A to point B; “information” questions assess the ability to search and find information such as sports events; and “command” questions are related to tasks like sending reminders of upcoming appointments.
Google Assistant is ranked the highest across all categories except “command.” Siri ranks higher in this category. This may due to the fact that Apple HomePod will pass any request relating to “messaging, lists and anything other than music to the iOS device paired to the speaker,” Loup Ventures says, adding that iPhone has deep integration with email, calendar, messaging, etc.
Among the smart assistants on the market, “the differentiator will be the real utility that comes from contextual awareness,” says Loup Ventures, adding that if you ask, “what’s on my calendar?” a truly useful answer may be, “your next meeting is in 20 minutes at Starbucks on 12th street. It will take 8 minutes to drive, or 15 minutes if you take the bus. I’ll pull up directions on your phone.”
It may suggest that Apple’s HomeKit smart home ecosystem will gain the upper hand, as “command” is the only question category among the five that may bear any relation to appliances control.
Alexa Loses edge on e-commerce
One would expect Amazon gets higher rank in the “commerce” category but Google Assistant correctly answer more questions about product information and where to buy certain items. And Google Express is just as capable as Amazon in terms of actually purchasing items or restocking common goods users have purchased, Loup Ventures says.
The firm provided an example. When asked the question “How much would a manicure cost?” Alexa replied: “The top search result for manicure is Beurer Electric Manicure & Pedicure Kit. It’s $59 on Amazon. Want to buy it?
Google Assistant, on the other hand, said, “On average, a basic manicure will cost you about $20. However, special types of manicures like acrylic, gel, shellac, and no-chip range from about $20 to $50 in price, depending on the salon.”
In the “local” and “navigation” categories, Google Home and HomePod outperformed the others by a distance, thanks to their integration with proprietary maps data, as many questions were related to local businesses, bus stations, names of towns, etc.
All voice assistants become smarter
Nevertheless, all the voice assistants have turned smarter compared with previous test results. Over a 12-month period, Google Home improved by 7 percentage points, Echo by 9 points, Siri (9 month) by 22 points and Cortana by 7 points in terms of questions answered correctly. “We continue to be impressed with the speed at which this technology is making meaningful improvement,” Loup Ventures says.
Alexa had the most noticeable improvement. The largest advancement came in the “information” section, where Alexa was much more capable with follow-on questions and providing things like stock quotes without having to enable a skill.