Piper Jaffray's Gene Munster asked an iPhone 1,600 questions, 800 on the busy streets of Minneapolis, 800 in a quiet room, in order to compare the performance of Google vs. Siri, reports FORTUNE.
Analyzing the results, Munster estimated that Siri is more than two years behind Google giving it a grade D compared to B+ for Google.
Notably, Siri uses Google to answer 60% of its queries. The rest of the answers are divided as follows: 20% of from Yelp, 14% from WolframAlpha, 4% from Yahoo and 2% from Wikipedia.
"Breaking down Siri's reliance further," Munster writes, "Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results."
RESULTS: ● Google understands 100% of the questions (not surprisingly, since they are keyed in) ● Google replies accurately 86% of the time ● Siri comprehends 83% of queries in noisy conditions, 89% in a quiet room ● Siri answers accurately 62% of the time on the street and 68% in a quiet room.
COMMON SIRI MISTAKES: ● What team does Peyton Manning play for? Responded with the answer to the previous query. This was the most common error. ● Where is Elvis buried? Responded I can't answer that for you. It thought the persons name was Elvis Buried. ● Where am I? Pin dropped in the wrong place. ● When did the movie Cinderella come out? Responded with a movie theater search on Yelp. ● How do I get from Boston to New York? Responded I can only give directions from your current location. I cant give you directions to a place you are not in. ● What spices are in Lasagna? Responded with a Yelp search with lasagna on the menu. ● When is the next Haley's comment? Responded "You have no meetings matching Haley's" ● I want to go to Lake Superior? Responded with directions to the company Lake Superior X-Ray.
Without seeing the kind of questions asked (not just the ones that Siri failed), it's impossible to evaluate this test. Siri's biggest problem, aside from having to transmit the questions to be translated before answering, is it's overreliance on commerial answers (see the "movie question").