Analytics firm Loop Ventures published the annual Smart Speaker IQ Test this week. The research firm put the top smart assistants and speakers head-to-head as part of the new test, grading them on a wide range of queries and commands.

The company asked Google Assistant, Amazon Alexa, Siri, and Cortana the same 800 questions. These questions were categorized by five different topics: Local, Commerce, Navigation, Information, and Command. The personal assistants were tested on their respective smart speakers — that includes the Google Home, Amazon Echo, Apple HomePod, and Harman Kardon Invoke (Cortana).

Google Assistant beat the other assistants in almost all these categories, taking the first place in the test. The assistant answered 87.9% questions correctly, understanding 100% of all the 800 queries. Siri came in second, answering 74.6% of the queries correctly and understanding 99.6% of the queries. Alexa came in third, answering 72.5% of the queries correctly and understanding 99% of the queries.

And then there’s Cortana, coming in last with 63.4% accuracy. Cortana actually understood more queries accurately than Alexa, only getting 5 queries wrong as opposed to Alexa’s 8 misunderstood queries.

In terms of the different categories, Google Assistant took the crown in everything except the Command category where Siri on the HomePod took the crown, which is kind of interesting. What’s even more interesting is that Google Assistant and Siri actually beat Amazon’s Alexa on the Commerce category, which is a big surprise considering the size of Amazon.com.

Overall though, all of these assistants have improved over the last year. Cortana only had a 56% accuracy a year ago, so the new 63.4% is quite an achievement, even though it’s still at the last position. In fact, Siri took a huge step forward over the last year — back in February 2018, Siri was sitting at 52%, and it’s now at a whopping 74.6%, taking over both Alexa and Cortana.

Tagged with Alexa, Cortana, Google Assistant, Siri, Smart Speaker IQ Test