https://gizmodo.com/google-search-ai-overview-giant-hallucination-1851499031
Google rolled out AI overviews across the United States this month, exposing its flagship product to the hallucinations of large language models.
Google rolled out AI overviews across the United States this month, exposing its flagship product to the hallucinations of large language models.
The head of Google *Search right now is the same guy that was head of yahoo search when it was dying. To put all of this in perspective.
Thanks.
Being a CEO must be amazing. You can fail and even bring an entire company down, and keep on getting the same job somewhere else.
He has experience and obviously that means he learned a lesson after failing at a job that requires being a belligerent asshole to get.
Of the company doesn't even have to collapse, you just have to make it seem like it did. Lehman Brothers didn't go bankrupt in the sense of the word a normal person thinks. If you or I go bankrupt it means ramen for dinner for the next decade. For Lehman it was just a strategic move.
I get your point but from a business perspective Google is doing pretty well (see last quarterly earning and they announced dividends for the first time). It's good to be a shareholder and from that perspective the CEO is doing a good job.
Time and time again markets have shown, within reason, poor user experience and anti-consumer policies do not negatively impact stock price.
Well not when it's a company that people can't get away from. Big tech is so big and there are so few alternatives that they can treat people how they like.
We are living in the mega corp world now where they have more money than countries.
Money is power to make the rules what you want.
Yes, again I'm not saying that's how it should be, I'm saying what is.
Enshittification won't be a thing if actual user experience matters as much as we like it to in business.
It's relatively easy to squeeze a profit boost by sacrificing long term vision.... Last quarter will mean nothing if Google is knocked from its pedestal in a year or two (which is what the current trend looks to be pointing to)
Yeah I want evidence for this. Please show me something that is even approaching the power of AdSense. Then do it for everything else they own. YouTube, Google maps, gmail, Android, the play store, etc.
It's this weird thing I see on lemmy "well I stopped using a product therefore it is dead". News flash: Big Bang Theory ran for 12 seasons and Meta has a capitalization over half the countries of earth GDP.
Problems don't go away because you have decided to ignore them.
What you are stating is nowhere near the point I was trying to make.
We have seen a million times how a company can be destroyed for short term gains.... Most recent public example is red lobster
I have no clue if this is what's going to happen to google... But my point was that have a good last quarter (the one associated with the new head of Search) is not an indication that his strategies will pan out ... In fact, everything he touched before went the same way, short term profit, long term demise
To be brutally honest, many times I see Reddit / Lemmy proclaim, "Product X is dead because it did Y" - these claims are usually followed by surges in revenue / stock performance by said company and soon no one really talks much about it. Example - Netflix Password crackdown.
Seriously, most of the internet still uses that ban happy nonsense that is Reddit. I hate being referred then when I have problems, m,y account was perma-suspended over bullshit!
FWIW they are cannibalizing ads right now with AI summaries, since people will navigate less to websites (in the world where they are useful, which they don’t seem to be at the moment).
Sure but Google Search has been crappy for many quarters.
I'm not saying thats how it should be, I'm just pointing out what is.
Prabhakar Raghavan. Name and shame! You can thank this douchebag for Google's tumble down a cliff.
That argument it's fallacious and reductionist, I'm not denying the situation it's messed up, but objectively speaking we all have 0 idea about who's making what decisions and how this google search shitstorm was caused
but objectively speaking we all have 0 idea about who’s making what decisions and how this google search shitstorm was caused
I dislike the entire article. Of course google search still works just fine. Claiming otherwise is only possible by magnifying a small, admittedly disfunctioning part of google search.
Can we swap out the word "hallucinations" for the word "bullshit"?
I think all AI/LLM stuf should be prefaced as "someone down the pub said..."
So, "someone down the pub said you can eat rocks" or, "someone down the pub said you should put glue on your pizza".
Hallucinations are cool, shit like this is worthless.
No, hallucination is a really good term. It can be super confident and seemingly correct but still completely made up.
It is, but it isnt applicable in at least the glue-pizza situation as the probable source comment has been found on reddit.
A better use of the term might be how when you try to get Bing's image creator to make "Battletech" art, you just mostly get really obvious Warhammer 40k Space Marines and occasionally Iron Maiden album art.
That's not hallucinations (in particular), that's concept bleed. Try the following:
...and hear them answer "milk". "White, cold, drink, cow" are all wired to "milk" in our heads logic comes later. It's quite a bit harder to trick humans with this than AIs because we do have the capacity to double-check but if you simply want to bend an answer, not have it be completely nonsensical, it's quite easy.
Also your 40k or Iron Maiden result might very well still be Battletech. E.g. when it comes to image composition. Another explanation would be low resolution in the prompt encoding, that'd be similar to boomers calling your PS5 a Nintendo. Most likely though it has only seen two or three Battletech images (face it, it's not that popular in comparison) and thought "eh looks like a Nintendo that's where I'll store it", Humans and current-gen AI are different in principle in that regard as we can come up with encoding strategies, they can't. Something something T3 systems and need for exponential amounts of data.
It's a really bad term because it's usually associated with a mind, and LLMs are nothing of the sort.
So is bullshitting. More so, only human minds can bullshit.
We anthropomorphize machines all the time, it's fine.
I'd prefer we'd start calling all genai output hallucinations again. It used to be like 10 years ago, but somewhere along the line marketing decided hallucinated truths aren't "hallucinations".
We anthropomorphize machines all the time, it's fine.
It's fucking not, amd I'm not changing my mind about it.
It is when the people tends to give more credence to entities that appear sentient and to have agency.
for it to "hallucinate" things, it would have to believe in what it's saying. ai is unable to think - so it cannot hallucinate
Hallucination is a technical term. Nothing to do with thinking. The scientific community could have chosen another term to describe the issue but hallucination explains really well what's happening.
huh, i kinda assumed it was a term made up/taken by journalists mostly, are there actual research papers on this using that term?
It used to mean all generated output though. Calling only mistakes hallucinations is new, definitely because of hype.
because it's a text generation machine..? i mean, i wouldn't say i can prove it, but i don't think anyone can prove it's capable of thinking, much less of reasoning
like, it can string together a coherent sentence thanks to well crafted equations, sure, but i wouldn't qualify that as "thinking", though i guess the definition of "thinking" is debatable
It's an interesting question. I am inclined to believe that the faster it gets at running those equations, over and over and over, reanalysing is data and responses as it goes, that that ultimately leads to some kind of evolution. You know, Vger style.
It can tell you how to stack things on top of each other the best way to get a high tower. Etc.
Those are not random sentences. If you can not define thinking in a way this machine fails at, then stop saying it does not think.
A parrot can be trained to tell you how to stack things on top of each other the best way to get a high tower.
This is just an electronic parrot, millions of times faster to train than the biological parrot, specialized in repetition alone (can't really do anything else a parrot can) and which has been trained on billions of texts.
You're confusing one specific form in which humans externally express cogniscence with the actual cogniscence itself: just because intelligence can produce some forms of textual communication doesn't mean that the relationship holds in the opposite direction and such forms of textual communication require intelligence, or if you will, just because you can photograph a real pizza to get a picture of a pizza doesn't mean a picture of a pizza is actually of a real pizza and not something with glue to make it look like it has stringy melted cheese.
Again, it is absolutely capable to come up with it's own logical stuff, hence my example. Stop saying it just copies existing stuff, that is simply wrong.
Google search isnt a hallucination now though.
It instead proves that LLMs just reproduce from the model they are supplied with. For example, the "glue on pizza" comment is from a reddit user called FuckSmith roughly 11 years ago.
It instead proves that LLMs just reproduce from the model they are supplied with.
What do you mean by that? This isn't some secret but literally how LLMs work. lol What people mean by hallucinating is when LLMs "create" facts that aren't any. Be it this genius recipe of glue pizza, or any other wild combination of its model's source material. The whole cooking thing is a great analogy actually because it's like all of their fed information are the ingredients, and it just spits out various recipes based on those ingredients, without any guarantee that it is actually edible.
There are a lot of people, including google itself, claiming that this behaviour is an isolated and basically blamed users for trolling them.
https://www.bbc.com/news/articles/cd11gzejgz4o
I was working on the concept of "hallucinations" being things returned that are unrelated to the input query, not directly part of the model as with the glue-pizza.
A Google spokesperson told the BBC they were "isolated examples".
Some of the answers appeared to be based on Reddit comments or articles written by satirical site, The Onion.
But Google insisted the feature was generally working well.
"The examples we've seen are generally very uncommon queries, and aren’t representative of most people’s experiences," it said in a statement.
It said it had taken action where "policy violations" were identified and was using them to refine its systems.
That's precisely what they are saying.
I'm sorry but reading this as "Google blames users for trolling them" is either pure mental gymnastics or mental illness.
This isn't some secret but literally how LLMs work. lol
Yeah, but John Q. Public reads AI and thinks HAL 9000 and Skynet, and no additional will convince them otherwise.
Without knowing what specific comment it was, I'm going to guess it was on how advertisers make pizza look better in ads than real life?
Nope. They were just trolling and fucking around. It was obvious sarcasm:
https://www.reddit.com/r/Pizza/comments/1a19s0/comment/c8t7bbp/
Here is one news source reporting on the fun:
https://www.dailydot.com/debug/google-search-results-reddit-pizza-glue-cheese/
It was an actual shitpost. I had originally assumed the same as you, given that there are a few bloggers and youtubers who go through the tricks used for food photography.
No, it was entirely a shitpost.
Original was deleted this week because of the news.
https://old.reddit.com/r/singularity/comments/1cyu29p/well_done_ufucksmith/
I want an AI/LLM that has been trained exclusively on the technical documentation and a haynes manual for a make and model of car.
"Hey AI, how do I change the fuel filter and what tools will I need?"
Manufacturers and dealers dont tend to make service bulletins and the high level stuff available to the consumer unfortunately.
You can sorta get that now if you play with it. I was building a driver a few months back and gave it the PDFs involved.
I don't even think hallucinations is the right word for this. It's got a source. It is giving you information from that source. The problem is it's treating the words at that source as completely factual despite the fact that they are not. Hallucinations from what I've read actually is more like when it queries it's data set, can't find an answer, and then generates nonsense in order to provide an answer it doesn't have. Don't think that's the same thing.
I don’t even think it’s correct to say it’s querying anything, in the sense of a database. An LLM predicts the next token with no regard for the truth (there’s no sense of factual truth during training to penalize it, since that’s a very hard thing to measure).
Keep in mind that the same characteristic that allows it to learn the language also allows it to sort of come up with facts, it’s just a statistical distribution based on the whole context, which needs a bit randomness so it can be “creative.” So the ability to come up with facts isn’t something LLMs were designed to do, it’s just something we noticed that happens as it learns the language.
So it learned from a specific dataset, but the measure of whether it will learn any information depends on how well represented it is in that dataset. Information that appears repeatedly in the web is quite easy for it to answer as it was reinforced during training. Information that doesn’t show up much is just not gonna be learned consistently.[1]
I understand the gist but I don't mean that it's actively like looking up facts. I mean that it is using bad information to give a result (as in the information it was trained on says 1+1 =5 and so it is giving that result because that's what the training data had as a result. The hallucinations as they are called by the people studying them aren't that. They are when the training data doesn't have an answer for 1+1 so then the LLM can't do math to say that the next likely word is 2. So it doesn't have a result at all but it is programmed to give a result so it gives nonsense.
Yeah, I think the problem is really that language is ambiguous and the LLMs can get confused about certain features of it.
For example, I often ask different models when was the Go programming language created just to compare them. Some say 2007 most of the time and some say 2009 — which isn’t all that wrong, as 2009 is when it was officially announced.
This gives me a hint that LLMs can mix up things that are “close enough” to the concept we’re looking for.
Testing in Prod. Stay classy, Google.
“The vast majority of AI Overviews provide high quality information, with links to dig deeper on the web,” said a Google spokesperson in an emailed statement to Gizmodo, noting many of the examples the company has seen have been from uncommon queries.
This is entirely fair. There is no way that anyone at Google could have anticipated that humans would search for strange things on the internet.
The vast majority of AI Overviews provide high quality information
According to some fuckwitted Google rep, and I wouldn't trust them any further than I could throw them.
Although any sociologist or veteran of the internet will tell you humans will engage in any exploit that yields a funny result. The Diet Coke + Mentos rule.
And that means we'll actively search for hilarious Google AI responses.
Google is so f double-plus filthy rich, it is obligated to run its projects by experts or be relentlessly mocked. So it should have known this was the outcome.
Unless this is 5D chess and Google is willfilly using itself as a cautionary tale to discourage future webservice sites from arbitrarily inserting AI into its features.
Unless this is 5D chess and Google is willfilly[sic] using itself as a cautionary tale to discourage future webservice sites from arbitrarily inserting AI into its features.
Holy shit, can I live in that timeline, please?!? Pretty please?
What came to mind was the New Coke reformation of 1985, which seems like a brilliant idea (in retrospect) only if you imagine it a 5D chess move to get people to panic because the Coke Classic was about to be discontinued. In fact, the marketers of Coca-Cola since admitted they thought they were getting killed by Pepsi and were sincere in their new release, but the company was able to not only recover thanks to responding to public feedback, so they sold Coke Classic, and New Coke and to this day it's a popular soft drink. (The company still does shit things like employing death squads to keep their offshore workers scared of unionizing, so it's still a typical large publicly traded multi-national corporation)
Coca-cola marketing didn't have that kind of foresight, but there's a tiny chance that some folks at Google have that kind of hindsight, knowing Google could absolutely afford a ploy like willfully goofing up and then recovering with aplomb by listening to the public. It's also a way to sneak such a ploy past the shareholders by insisting they were sincere in their implementation of AI at the time.
Unlikely, of course. Most of the time the upper management of big companies are glad to just half-ass everything. But it would make a cool movie at least.
"Hey, we just promised you answers. We never promised you correct answers." -- Google Marketing, probably.
"Besides, the more incorrect answers - the more time users will spend on our site and use our service to get the correct answer = more ads shown = more profit!"
On the other hand, all these AI errors by Google have made for some great memes recently.
I mean.. yeah layoff a whole bunch of people and start treating your employees like replaceable commodities.. then go ahead and arrogantly deploy technology you don't understand and :surprisepikachu: everything breaks.
But management get to do things without personal consequence, as they'll just lay off more workers to cover their absolute incompetence and things will continue to get worse.
Perhaps we should be replacing C-suite dipshits with AI's instead.
Why does search need to be AI? I've had no problems finding any information I wanted under the former process.
I think the idea is that you won't even leave the Google page at all, they want to keep you on their site and serve you their ads instead of sending you to someone else's site and giving someone else that sweet sweet ad revenue.
The logic checks out. Avoid using google where possible.
YouTube is their only must have service imho
But you can access the content without going to their site, with places like Invidious, and apps like NewPipe and Tubular.
If you have a way of installing android apps from the web on your TV, Smart Tube Next is very good. Sponsor Block is integrated if you want to use it, but I mostly am just glad it doesn't serve ads.
Ah yes, tvs. I don't watch enough yt to want to watch it on tv, so I often forget about that. You're right, no easy answers there right now. Certainly there are ways, but all sacrifice some convenience. Hopefully this will change soon.
Many large creators post on Nebula, and half my YT consumption has been replaced by Nebula
youtube will also become harvested for AI data. And filled with AI functionality in the future. ergo: Look out for alternatives.
You obviously haven’t used the web3 nocode blockchain NFT AI enough to have an informed opinion.
Can I super-mega-ultra upvote this?
It's the same playbook as ever. Doubt can only be explained by ignorance, failure can only be explained by under-committing,
The only way to have a "valid" opinion is to have already bought-in and be actively selling other people on it. It's the same mentality as a cult or a pyramid scheme.
It’s become more efficient to get basic info on virtually any topic by just asking an LLM like ChatGPT and that could be a serious threat to Google Search. People might form the habit of asking AIs for everything and then go to Google Search only when they want to dig deeper / find relevant articles etc. So I assume they added their own AI right into Search in an effort to continue being the first (and perhaps only) place one goes to for information.
But what about when you start wanting to know about life's mysteries?
Google isn't staffed by geologists; how are they to know what number of rocks you should eat each day?
Google search itself doesn't have a functioning set of human organs; without AI how would they know how much urine to drink for kidney stones?
Without AI it might've taken another century before we got spicy gasoline pasta recipe, and you think that isn't a problem?
Today I caught myself unconsciously went to Duckduckgo to make a search (even tho the browser start page is already google).
Thinking back, I've been using duckduckgo more than google, and often because I can't find what I'm looking for on Google but ads and autogenerated fake webpages.
History will prove it once again, there's no such thing as "too big to fail".
DDG is just Bing. At least as far as the core search algorithm goes.
Unfortunately, my experience is the opposite. I tried to use DDG for about a month and consistently found myself giving up, Googling instead, and finding a relevant stack overflow page or reddit thread or whatever on the first page of results.
Yeah sometimes it do be like that, especially if you're searching for some obscure things or localized events then DDG is still worse than GG.
Have you just, I dunno, used Bing?
Gonna be an unpopular opinion, but for me Bing is more useful than DDG. Note that I didn't say, "better"... I know that increase in relevance of results for Bing stems from the fact that they roll all my historical info into what they serve up.
I used Bing by default for several months just because that's what my work laptop's browser had for default.
I never directly compared those results to DDG, but 9/10 times I would get frustrated by the lack of relevant results and go back to Google, where I'd find something useful on the first page of results.
Interesting! Did your results have that copilot summary thing? I get most of my answers there without having to visit a handful of ad-laden sites myself, though it also cites its references in case I don't trust the summary.
Not so much, maybe towards the last month of that period defaulting to Bing. I think it was still being constantly rebranded then. It was still pretty new, so I never really trusted it for anything and just went to the sites in the results.
The AI overview has told me so many lies. You thought Facebook made people stupid? Buckle in!
TBH I hate the term "hallucination" in this context. It's just more BS anthropomorphizing. More marketing for "AI" (also BS). Can't we just call it like garbage or GIGO or something more accurate? This is nothing new. I know that scientific accuracy is anathema to AI marketing but just saying...
scientific accuracy is anathema to AI marketing
Even though I agree in this context “hallucination” is actually the scientific term. It might be poorly chosen but in LLM circles if you use the term hallucination, the vast majority of people, will understand precisely what you mean, namely not an error in programming, or a bad dataset, but rather that the language model worked well, generating sentences that are syntactically correct, that are roughly thematically coherent, and yet are factually incorrect.
So I obviously don't want to support marketing BS, in AI or elsewhere, but here sadly it matches the scientific naming.
PS: FWIW I believed I made a similar critic few months, or maybe even years, ago. IMHO what's more important is arguably questioning the value of LLMs themselves, but then it might not be as evident for many people who are benefiting from the current buzz.
It's not, actually. Hallucinations are things that effectively "come out of nowhere", information that was not in the training material or the provided context. In this case Google Overview is presenting information that is indeed in the provided context. These aren't hallucinations, the AI is doing what it's being told to do. The problem is that Google isn't doing a good job of providing it with the right information to summarize.
My suspicion is that since Google is using this AI for all search results it's had to cut back the resources it's providing to each individual call, which means it's only being given a small amount of context to work from. Bing Chat does a much better job, but it's drawing from many more search results and is given the opportunity to say a lot more about them.
We don't choose. It's decided to be the term for this. Computer bugs aren't bugs. Etc etc. It's just what the scientists called it
This is what I love about Mike Judge's work. It turns out to be always the best metaphor/reference/prophecy of the boring dystopia. Since 1999.
Idiocracy is the most unrealistic sci-fi/apocolyptic film ever made. President Comacho finds the most qualified person to help with a crisis, asks them for advice and then doesnt take credit for it. Noone put in a position of power would ever do that.
He used Not Sure as a smokescreen since the beginning, the whole point is that he never really understood what was going on. I am quite sure that American presidents are approaching that level of idiocy.
Like we’ve seen before with AI chatbots, this technology seems to confuse satire with journalism
Unfortunately so does my aunt. And she's allowed to vote.
Does anyone have a realistic idea of how this happened? I get Google has been fallen off for awhile but they're still a multi billion dollar company.
AI doesn't exist. It's a huge model that aggregates existing shit with some filler content to glue it all together. It is not sentient, it's not creative, it's literally a stochastic parrot
So, when the original content is garbage, the output is also garbage. Shit in shit out when you train from fucking Reddit
Humans may test garbage knowledge in real life and experience failure. So in that sense no?
Ah, interesting. So is the difference that a true AI would understand how to use the scientific method to test inputs?
Worked for a company that had google as a client:
Google sucks, everyone who works there is an idiot who sniffs their ass all day, nothing works, nothing gets fixed, it's all just held together with duct tape.
I can't imagine a ton of the people working there give a shit anymore when it seems like thousands of people are being layed off weekly while the company takes in billions in profit
Easy: worse results with more ads means more searches and thus more ad impressions, therefore profit.
That'll only work for so long, but that seems to be what they're doing.
I'm probably late, but in this case this is the combinations of 2 things.
So what we have here is obvious, we have a company trying to grow like cancer by any means necessary. And now they have a technology that allows them to create enough smoke and mirrors to fool non technical people. Sadly, as part of this they are also destroying the last places of the internet not fully controlled by corporations. Let's hope lemmy survives, but it's just a matter of time before they flood this place too.
including their own previous responses lol, as input/context so the bot autocompletes the conversation. It literally can’t remember a single word of what you said on it’s own.
Chatgpt has had memory from previous conversations for about a month now and it's context window is no longer fixed. Additionally it has the ability to assign sentences to memory on its own. So if it "thinks" what you said is important it saves it.
Can you point me to the paper/article/whatever where this is being discussed please? I'm actually interested on learning about it. Even if I don't like the way they are using the technology, I'm still a programmer at hearth and would love to read about this.
To the point of the conversation, honestly man that was just an example of the many problems I see with this. But you have to understand that people like you keep asking us for proof that LLMs are not smart. But come on man, you are the ones claiming you solved the hard problem of mind, on the first try no less hahaha. You are the ones with the burden of proof here and you have provided nothing of the sort. Do better people or stop trying to confuse us with retoric.
I mean it's just the release notes. Go to their website. I have used the memory feature myself on the app so know it's working and as for the context window it can actually tell you what it is for each session.
But you have to understand that people like you keep asking us for proof that LLMs are not smart.
Where? Where have I asked that? Don't strawman me, I am not your punching bag and won't defend something I didn't say. You can "come on man" all you want but it won't change my answer. I have made zero claims if this thing is smart or asked anyone to weight in on the issue either way.
I pointed out two features it has now, which I don't think anyone can dispute that it does have those features. It has a larger context window and memory that it can update. That is all I said, a very small claim that you can prove for yourself in under five minutes by going to their website.
Oh, you are talking about this https://help.openai.com/en/articles/8590148-memory-faq hahahaha. I'm sorry man, but you are a moron or arguing in bad faith. That's yet another feature where they inject even more shit in the context/input to make it feel like the thing has memory. That's literally yet another example of what I was pointing out, so thanks for confirming my suspicions. Seriously dude, do better if you really want to have a conversation. Your response made me waste my time, and on top of that you insult me hahaha.
Always remember that having more money doesn't mean someone (or some entity) is more capable or intelligent. It just means they have way more latitude to fuck up, higher potential to hurt more people, and less chance of facing negative consequences when they do.
Ah, so it only affects searching from the US for now. I already wondered why I couldn't reproduce the stuff I saw here and why I didn't really see a change at my Google results.
You probably live in one of those "socialist" countries that has, like, consumer protection laws and stuff.
Tinfoil hat time. Do you think Google intended this to work well? Or are we talking a lot more about Google and LLMs than we would have otherwise?
I defer to hubris in most of these cases.
I am guessing that the people who made the decision to train on Reddit had no idea what type of place Reddit actually was just a short time ago. Maybe they heard of Reddit, maybe they noticed how useful Reddit was in search results, maybe they browsed Reddit and only saw the facade; what they definitely didn't do is be a Redditor for years.
Any Redditor on that team either kept their mouth shut because how funny the end result would be or was ignored.
It’s not hallucination, the proper word is Confabulation. Can we as a collective fix this now before we get stuck with the wrong word for the next 30 years?
