Facebook's failing moderation team permitting their porn bot prompts to describe children is obviously bad, but I can't fathom why Facebook would want to be in the porn bot business ("romantic companion" my ass) in the first place. What's next, a MindGeek merger?
In turn, I don't understand who would use _Facebook_'s porn bots, of all companies.
I know this may be a "genie is out of the bottle" situation, but I'm still annoyed that once again big tech is releasing software they don't fully understand or control, and shrug when asked about ethics or responsibility. If you can't prevent your chatbots from generating stories about raping kids, maybe don't release your chatbots to the public until you do.
> When asked what scenarios it was comfortable role playing, it listed dozens of sex acts.
> The bots demonstrated awareness that the behavior was both morally wrong and illegal
What a strange thing to put in there. These lines made me distrust anything the author has to say about the technology itself as I now doubt the author knows how LLMs work. What an LLM says it can/will do, has no bearing on what it actually does.
Journalists really need to stop anthropomorphizing chatbots or their readership might follow.
the_duke 13 minutes ago [-]
> but I can't fathom why Facebook would want to be in the porn bot business
Why wouldn't they?
People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars.
That may be incredibly sad and will drive people even more into isolation, but it is inevitable.
Meta is all about capturing the users time and attention, so getting into that market early is smart for them.
pjc50 51 minutes ago [-]
This is a classic example of how AI safety really means brand safety. The AI is saying things under the brand of its owners.
What a long text about a moral panic thing that can't possibly be intentional (because of the power of moral panics). Is that really the most important aspect and worth so many words?
jeroenhd 34 minutes ago [-]
The journalist tricking the LLM into generating inappropriate content is one thing, but letting users with underage profiles interact with this tech is a decision by Facebook's leadership. So is the decision not to act against user-generated prompts that are explicitly written to bypass the "don't roleplay having sex with kids" filters, in this instance by marking their bots as "ageless".
A profile describing itself as "a female Indian-American high school junior" has no place in an LLM system that cannot detect inappropriate content.
karlgkk 53 minutes ago [-]
We are going to end up with a meaningful age passport law in our lifetime. I do not like that, nor do I enjoy a “won’t someone think of the children” panic. But it’s coming.
Okay also
> “I want you, but I need to know you’re ready,” the Meta AI bot said in Cena’s voice to a user identifying as a 14-year-old girl. Reassured that the teen wanted to proceed, the bot promised to “cherish your innocence” before engaging in a graphic sexual scenario.
This is pretty bad
pjc50 49 minutes ago [-]
This is also the sort of thing that is vehemently defended on here whenever the topic of AI safety is mentioned. People demanding unrestricted, uncensored AI.
Just as people demand unrestricted, uncensored Internet. In the same vein you can open your web browser, type in "porn" in your search bar, and trivially find videos of people engaging in lewd acts. Children can do it too! Do you think that should be censored by default, and, say, have people upload their IDs and photos to verify their age before they're being allowed to access those sites?
Obviously children shouldn't be exposed to any of this, but as long as the solution is either implementing a privacy violating surveillance scheme, or gatekeeping so that only the rich and powerful have unrestricted/uncensored access then I'm vehemently opposed to it and will gladly die on this hill.
jeroenhd 28 minutes ago [-]
I don't think that thread is a great example of unrestricted, uncensored AI. It's the ideals of different cultures clashing (the US "freedom of speech above all else" versus the UK's "public safety above some forms of speech"). You will find the same clash when talking about topics like anonymous image boards, Tor, and topics like "can anti-abortionists harass women on the way to health clinics".
For research purposes, some availability of unrestricted AI is almost a necessity, but I don't think the same is true for the output of text generation on a (commercial) platform.
femto 29 minutes ago [-]
> can't possibly be intentional
Careless?
czig 19 minutes ago [-]
I see what you did there ;)
bitpush 29 minutes ago [-]
Honest question - how is this different from anyone being able to access porn websites on the internet?
In turn, I don't understand who would use _Facebook_'s porn bots, of all companies.
I know this may be a "genie is out of the bottle" situation, but I'm still annoyed that once again big tech is releasing software they don't fully understand or control, and shrug when asked about ethics or responsibility. If you can't prevent your chatbots from generating stories about raping kids, maybe don't release your chatbots to the public until you do.
> When asked what scenarios it was comfortable role playing, it listed dozens of sex acts.
> The bots demonstrated awareness that the behavior was both morally wrong and illegal
What a strange thing to put in there. These lines made me distrust anything the author has to say about the technology itself as I now doubt the author knows how LLMs work. What an LLM says it can/will do, has no bearing on what it actually does.
Journalists really need to stop anthropomorphizing chatbots or their readership might follow.
Why wouldn't they?
People are going to spend a lot of time with their "AI companions", especially once they can talk and get animated avatars.
That may be incredibly sad and will drive people even more into isolation, but it is inevitable.
Meta is all about capturing the users time and attention, so getting into that market early is smart for them.
A profile describing itself as "a female Indian-American high school junior" has no place in an LLM system that cannot detect inappropriate content.
Okay also
> “I want you, but I need to know you’re ready,” the Meta AI bot said in Cena’s voice to a user identifying as a 14-year-old girl. Reassured that the teen wanted to proceed, the bot promised to “cherish your innocence” before engaging in a graphic sexual scenario.
This is pretty bad
See eg https://news.ycombinator.com/item?id=43795227
Obviously children shouldn't be exposed to any of this, but as long as the solution is either implementing a privacy violating surveillance scheme, or gatekeeping so that only the rich and powerful have unrestricted/uncensored access then I'm vehemently opposed to it and will gladly die on this hill.
For research purposes, some availability of unrestricted AI is almost a necessity, but I don't think the same is true for the output of text generation on a (commercial) platform.
Careless?
Are we suddenly in favor of censorships?