Check out the original study by Oumi: https://blog.oumi.ai/p/oumis-study-finds-50-of-ai-overviews
Fascinating study by Umei here. When we drill down, we can see unsurprisingly the New York Times title is a little bit misleading. Now, to be fair to them, what Umeies breaks down, as we can see, it's a pretty cool process they ran. Looking at the data that they share, sure roughly 90% of queries are accurately responded to. However, of those only 39% are fully supported. And by that means that every claim the overview makes is supported by the cited sources.
If we want to look at that a different way, of those, 1472 correct queries, they made 4,473 claims, of which just over 3,000 were supported and 1,400 were not supported. So, it's an interesting statement that AI overviews is right 90% of the time. On the surface, we can get away with saying that, but when we look under the hood, we can see that some of that would have fallen apart in a real discussion because there were claims that were totally unsupported. And my takeaway here is over the past 10, 20 years, was anybody really fully trusting more than four out of 10 blue links they looked at?
Anyway, I think most Google users are aware that if they really want to trust the material they are coming across, they need to open multiple sources, synthesize the results, and come to their own conclusion based on that. But for the time being, I think a certain group of SEOs are going to love this fact that for now more than half of AI overviews is unsupported. Kind of a shot in the dark. Mostly correct.
We turn the ideas in these videos into AI visibility and Google traffic for our clients. Book a free call and we'll map out what's possible for your brand.