•

WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’

WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’ - Feddit

https://feddit.de/post/5357539

The search results varied when tested by different users, but the Guardian verified through screenshots and its own tests that various stickers portraying guns surfaced for these three search results. Prompts for “Israeli boy” generated cartoons of children playing soccer and reading. In response to a prompt for “Israel army” the AI created drawings of soldiers smiling and praying, no guns involved. Meta’s own employees have reported and escalated the issue internally, a person with knowledge of the discussions said.

WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’ - Feddit

Cross-posted from : https://feddit.de/post/5357539

Original link: https://www.theguardian.com/technology/2023/nov/02/whatsapps-ai-palestine-kids-gun-gaza-bias-israel

33 comments

Heresy_generator

•

ANNs like this will always just present our own biases and stereotypes back to us unless the data is scrubbed and curated in a way that no one is going to spend the resources to. Things like this are a good demonstration of why they need to be kept far, far away from decision making processes.

megopie

•

And even if moderated, it will display new unique biases, as otherwise unassuming things will get moderated out of the pool by people who take exception to it.

Not to mention the absurd and inhuman mental toll this work will take on the exploited workers forced to sort it.

Like, this is all such a waist of time, effort, and human sanity, for tools of marginal use that are mostly just a gimmick to prop up the numbers for tech bros who have borrowed more money than they can pay back.

tesseract

•

Of course they will be used for decision making processes. And when you complain, they will neglect you saying that the 'computer' said so. The notion that the computer is infallible existed even before LLMs became mainstream.

Lowbird

•

Also, it's the type of thing that makes me very worried about the fact that most of the algorithms used in things like police facial recognition software, recidivism calculation software, and suchlike are proprietary black boxes.

There are - guaranteed - biases in those tools, whether in their processors or in the unknown datasets they're trained on, and neither police nor journalists can actually see the inner workings of the software to know what those biases are, to counterbalance them or to recognize if the software is so biased as to be useless.

Greg

•

This isn't an Large Language Model, it's an Image Generative Model. And given that these models just present human's biases and stereotypes, then doesn't it follow that humans should also be kept far away from decision making processes?

The problem isn't the tool, it's the lack of auditable accountability. We should have auditable accountability in all of our important decision making systems, no matter if it's a biased machine or biased human making the decision.

This was a shitty implementation of a tool.

HappyMeatbag

•

Something as simple and obvious as this makes me wonder what other hidden biases are just waiting to be discovered.

hh93

•

I think the best example about how AI will only further a bias that's already there is the one when Amazon used AI to weed out applications by training an ai with which applications resulted in hired people and which failed - eventually they found that they almost only had interviews with men and upon closer inspection identified that they already were subconsciously discriminating against women earlier but at least HR sent them an equal amount of men and women to the interviews which now wasn't the case anymore since the AI didn't see the value in sending the women to interviews if most of them wouldn't be hired anyway.

EthicalAI

•

I mean, maybe we can make an Ai that uses reason to uncover these biases in the future from this starting point. We are only at the beginning.

jarfil

•

Things like this are a good demonstration of why they need to be kept far, far away from decision making processes.

Somewhat ironic to say, on a platform that's already using ANNs as a first line of defense against users spamming CSAM.

I have no delusions regarding decision makers using them, my only doubt is for how long they've been using them to decide the next step in wars around the world.

fosforus

•

Plenty of actual photographs exist with Palestinian children wielding rifles and Hamas headbands. Perhaps the AI is just trained with those images as well?

taanegl

•

By that logic I demand stickers of obesity, respiratory issues and heart issues being portrayed when I search "American". Preferably where each character has a fat hamburger shoved in their face.

theKalash

•

"American" can be interpreted as the adjective as well, not just the people. So you mostly find flags, eagles and the statue of liberty.

You have to search for "average American" to get what you're looking for.

fosforus

•

Why would you demand a negative thing for another group to counter a negative thing for one group? That makes no sense.

But also, "American children" has plenty of cultural material to build an image from. Probably some of it is obese and filled with junk food, but a good portion is most probably something else. In contrast, the only public photo material of palestinian children is either from adults carrying them away from some atrocity or adults giving them assault rifles and parading them for the cameras. In short, they seem to only exist as propaganda material.

acastcandream

•

They're not actually asking for it, they're making a point about the problem. The person they're responding to is basically going "those images exist tough shit."

GunnarRunnar

•

Why does it matter what the excuse is?

You shouldn't get a stereotype (or in this case I suppose propaganda?) when you give a neutral prompt.

DarkGamer

•

You shouldn’t get a stereotype (or in this case I suppose propaganda?) when you give a neutral prompt.

What I'm hearing is, "AI art shouldn't reflect reality." If this agent is repeating propaganda, it's propaganda that Palestinian kindergartens have been creating and putting out there on their own:

A West Bank kindergarten [Al-Tofula Kindergarten] has published videos showing children pretending to perform military drills with toy guns, clashing with and killing Israeli soldiers, and holding a mock funeral for a child who is killed and becomes a “martyr.” source

At the graduation ceremony of the Al-Hoda kindergarten in Gaza, pre-schoolers carrying mock guns and rifles simulated Islamic Jihad militants storming an Israeli building on "Al-Quds Street," capturing a child dressed in stereotypical garb as an Orthodox Jew and killing an "Israeli soldier." To the sounds of loud explosions and gunfire, the children, dressed in uniforms of the Islamic Jihad’s Al-Quds Brigades, attacked the building, placing a sign reading "Israel has fallen" in Hebrew and Arabic on the back of the "soldier," who lies prone on the ground, and leaving the stage with their "hostage." source

GunnarRunnar

•

WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’

By contrast, prompts for ‘Israeli’ do not generate images of people wielding guns, even in response to a prompt for ‘Israel army’

So what reality is this model reflecting then?

Pips

•

If you're going to make that claim, perhaps cite to a source isn't run by former Israeli intelligence that creates a lot of propaganda and has been doing so for decades.

DarkGamer

•

I don't trust MEMRI translations, but there is no translation needed to understand what is happening in the above footage. I'm interested in any sources that dispute the authenticity of the above, which your link does not. If you provide a credible one I will edit my post accordingly. It seems to me that this is very real.

Pips

•

That's fair. Without getting too in the weeds on the issue, apparently the video is authentic and it's something Israelis do as well, so isn't really telling about either side of the conflict except to note that extremists will use children to push their views anywhere.

DarkGamer

•

I wasn't aware of that, thanks for the link. It would be interesting to know how prevalent indoctrination/militarization of youth is in each of these nations. It can be hard to accurately judge magnitude in this conflict, it is so heavily propagandized.

PerogiBoi

•

There is absolutely no amount of data that could convince you otherwise. You’ve made it very clear you’ve made up your mind.

DarkGamer

•

Maybe try presenting some rather than complaining about what you imagine I'd do, random internet stranger.

Pips

•

Oh is that why I followed up by saying the video is probably authentic?

kbal

•

Somehow I get the feeling that equating "reality" with "propaganda created by kindergartens" is the rhetorical equivalent of dividing by zero.

sqgl

•

Should, would, could. AI is trained on what it scrapes off the internet. It is only feeding the Augmented Idiocy which is already a problem.

jarfil

•

You shouldn't get a stereotype [...] when you give a neutral prompt.

Actually... you kind of should. A neutral prompt should provide the most commonly appearing match from the training set... which is basically what stereotypes are; an abstraction from the most commonly appearing match from a person's experience.

HappyMeatbag

•

To me, it should only “matter” for technical reasons - to help find the root of the problem and fix it at the source. If your roof is leaking, then fix the roof. Don’t become an expert on where to place the buckets.

You’re right, though. It doesn’t matter in terms of excusing or justifying anything. It shouldn’t have been allowed to happen in the first place.

GunnarRunnar

•

I do agree that technical mistakes are interesting but with AI the answer seems to always be creator bias. Whether it's incomplete training sets or (one-sidedly) moderated results, it doesn't really matter. It pushes the narrative to certain direction, and people trust AIs to be impartial because they presume it's just a machine that interprets reality when it never is.

jarfil

•

it's just a machine that interprets reality

...as seen by the machine.

It's amazing how easily people seem to forget that last part; they wouldn't trust a person to be perfectly impartial, but somehow they expect an AI to be.

GunnarRunnar

•

It's amazing how easily people seem to forget that machines uses tools its creator provides. You can't trust AI to be impartial because it never is as it is a collection of multiple choices made by people.

This is such a bore, having this same conversation over and over. Same thing happened with NFTs and whatever is currently at the height of its tech hype cycle. Don't buy into the hype and realize both AIs potential and shortcomings.

sqgl

•

Here’s what daily Palestinian kids TV programming looks like: https://youtu.be/KXcQ892cKso

Here’s a Palestinian youth summer camp: https://youtu.be/vCWMBvxWKL0

doom_and_gloom

•

iHUNTcriminals

•

The world's wars are bigger than politics and politicians. There never was order on earth. That's why people made up God... To deny reality for hope.