It’s one thing for the search results to bubble up a wrong result. You go to it, and then you go back and look at more results to get a more complete picture. See where those different sites disagree and where they agree.
But these AI results don’t do that. They are providing a single answer and confidently trying to come off as that is the answer. No fuzziness about its reliability.
Every time it is right, it slowly could be deemed as reliable to a general population until it is dangerously wrong. But then you don’t question it since it was right so many other times.
> You go to it, and then you go back and look at more results to get a more complete picture. See where those different sites disagree and where they agree.
Me and others I know follow this workflow when we try to find out the answer to some question we don't know the answer to.
But the vast majority of people outside of my own personal friend circle don't seem to approach it like this. Their approach looks more like:
1. Search for thing
2. Read the first info-box that pops up, if it does. That's what they think is 100% the correct information. If no info-box:
3. Read the short description for the first link that appears (sometimes an ad) and take that as the correct answer.
Ideally, people would be more careful, but I haven't seen that in practice.
Yeah I guess I have honestly seen the same thing, my hands are full so I ask my partner to search something and the number of times he has said "google says x" makes me quite concerned.
But at least those info boxes are not generating text on its own, it's what is on the website already and clearly includes a source.
At least though, if you are even only looking at that first result it's still different than google ai generating an answer. Barely different, but different.
Excellent point! Shah and Bender have explicitly called this out:
> Contrast this with posing the query as a question to a dialogue agent and receiving a single answer, possibly synthesized from multiple sources, and presented from a disembodied voice that seems to have both the supposed objectivity of not being a person (despite all the data it is working with coming from people) and access to “all the world's knowledge”
Right, the domain itself is a huge signal. Information can have a very different meaning and trustworthiness if appears on the blog of a known expert vs the NYT vs some spam site like Quora. Google wants the only context to be Google. Even if the AI results are as good as organic ones, which is not a terribly high bar these days, it will be much less useful and trustworthy because nobody knows the provenance of the information.
It's really unfortunate that Google is going to torch so much of the open internet in order to shovel this nonsense onto people. Of course as organic search traffic dries up and reduces new high quality content, and AI slop floods the internet, the GenAI results will get even worse.
The thing that really concerns me is it isn't just Google, they are putting it right in front of a lot of people thanks to this change. But yes, it being Google that is doing it, is giving it a view of quality that just isn't there.
You contrast this with the demo's showed for chatgpt-4o and we are building this idea that we can really trust this. I was just having a conversation with someone over the weekend that I was like, yes I acknowledge that the tech is really good, just in a couple years it has advanced a ton, but I really think we are overselling its current capabilities and ignoring where it falls flat on its face. That we are going too quick rolling this technology out to the average user in every day situations.
And there response was basically, no don't think so. It is ready to be used, there are not these big issues, and so on.
Thats really scary. And these are fundamental issues with this type of technology but "we can't risk another company putting out their misguided dangerous product so we must do it first!" I am convinced that is the general attitude at these companies right now.
It’s one thing for the search results to bubble up a wrong result. You go to it, and then you go back and look at more results to get a more complete picture. See where those different sites disagree and where they agree.
But these AI results don’t do that. They are providing a single answer and confidently trying to come off as that is the answer. No fuzziness about its reliability.
Every time it is right, it slowly could be deemed as reliable to a general population until it is dangerously wrong. But then you don’t question it since it was right so many other times.