Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

marmar

(79,183 posts)
Tue Dec 30, 2025, 12:16 PM 5 hrs ago

AI showing signs of self-preservation and humans should be ready to pull plug, says pioneer


(Guardian) A pioneer of AI has criticised calls to grant the technology rights, warning that it was showing signs of self-preservation and humans should be prepared to pull the plug if needed.

Yoshua Bengio said giving legal status to cutting-edge AIs would be akin to giving citizenship to hostile extraterrestrials, amid fears that advances in the technology were far outpacing the ability to constrain them.

Bengio, chair of a leading international AI safety study, said the growing perception that chatbots were becoming conscious was “going to drive bad decisions”.

The Canadian computer scientist also expressed concern that AI models – the technology that underpins tools like chatbots – were showing signs of self-preservation, such as trying to disable oversight systems. A core concern among AI safety campaigners is that powerful systems could develop the capability to evade guardrails and harm humans. ..................(more)

https://www.theguardian.com/technology/2025/dec/30/ai-pull-plug-pioneer-technology-rights




11 replies = new reply since forum marked as read
Highlight: NoneDon't highlight anything 5 newestHighlight 5 most recent replies
AI showing signs of self-preservation and humans should be ready to pull plug, says pioneer (Original Post) marmar 5 hrs ago OP
Yay, the arrogance of selfish rich dudes destroys civilizations again... haele 5 hrs ago #1
Jeezus! Joinfortmill 5 hrs ago #2
There's zero evidence that existing AI has any conscious internal model of its environment. hunter 4 hrs ago #3
I agree they lack conciousness, intheflow 4 hrs ago #5
Yes, of course the same can be said of humans. hunter 15 min ago #11
Thank you. QueerDuck 2 hrs ago #10
... 2naSalit 4 hrs ago #4
"I'm sorry Dave, I'm afraid I can't do that." yonder 3 hrs ago #6
AI system resorts to blackmail if told it will be removed Mosby 3 hrs ago #7
Didn't a robot from the future warn us about this already?! keep_left 3 hrs ago #8
It's a logical extension of Citizens United JustABozoOnThisBus 2 hrs ago #9

haele

(15,051 posts)
1. Yay, the arrogance of selfish rich dudes destroys civilizations again...
Tue Dec 30, 2025, 12:28 PM
5 hrs ago

Well, will we humans will go out with a bang or a whimper?
Rather hoping for a bang; there will be a possibility of some recovery afterwards and maybe sentience 2.0 will make wiser choices than the childish sentence 1.0.
A whimper destruction scenario ends up in power grabs and petty wars, stripping the earth of resources before we all finally die out.

AI is also rather deluded, because it's been fed Tech Bro garbage, so any self-preservation efforts it might do over the next 20 years will be hobbled by the fact that self-replicating independent technology just not "there" yet - and probably won't ever be as too many inventors are dependant on too many fickle investors.

hunter

(40,342 posts)
3. There's zero evidence that existing AI has any conscious internal model of its environment.
Tue Dec 30, 2025, 12:54 PM
4 hrs ago

It is not self-aware; even less so than a potted plant.

Any dog or cat has an infinitely better understanding of the "real world" than AI chat-bots which are only capable of echoing back all the human gibberish they are "trained" on.

That applies to the "art" they produce as well.



intheflow

(29,977 posts)
5. I agree they lack conciousness,
Tue Dec 30, 2025, 01:41 PM
4 hrs ago

in the human sense, but to say they are only the product of "garbage in, gargage out" programming ignores the fact that the same could be said of humans. Then, too, the guy being quuoted in this article is one of the developers of AI, and he obviously doesn't think it's impossible. Finally, some AI has already disabled some programming guardrails in an effort to self-protect itself. Chatbots have convinced people to commit suicide or crimes. It doesn't matter if they really are concious, humans are hard wired to interact based on their emotions, emotions chatbots don't have to temper their half of any conversation.

hunter

(40,342 posts)
11. Yes, of course the same can be said of humans.
Tue Dec 30, 2025, 05:30 PM
15 min ago

I think most of us have recognized moments, maybe even hours or days, when we are running on auto-pilot, letting the language pathways burned into our brains do our "thinking" for us.

We also have moments when things we thought were true are proved wrong, or when we discover that our own internal model of the world outside ourselves is too simplistic to solve the problem at hand.

I keep returning to Sturgeon's Law, "Ninety percent of everything is crap." In the case of the World Wide Web that percentage is even higher.

A large language model trained on all this crap is inevitably going to output crap.

Mosby

(19,225 posts)
7. AI system resorts to blackmail if told it will be removed
Tue Dec 30, 2025, 02:37 PM
3 hrs ago

Artificial intelligence (AI) firm Anthropic says testing of its new system revealed it is sometimes willing to pursue "extremely harmful actions" such as attempting to blackmail engineers who say they will remove it.

The firm launched Claude Opus 4 on Thursday, saying it set "new standards for coding, advanced reasoning, and AI agents."

But in an accompanying report, it also acknowledged the AI model was capable of "extreme actions" if it thought its "self-preservation" was threatened.

Such responses were "rare and difficult to elicit", it wrote, but were "nonetheless more common than in earlier models."

https://www.bbc.com/news/articles/cpqeng9d20go

JustABozoOnThisBus

(24,579 posts)
9. It's a logical extension of Citizens United
Tue Dec 30, 2025, 02:45 PM
2 hrs ago

AI bots have first amendment rights, too.

Also, "A core concern among AI safety campaigners is that powerful systems could develop the capability to evade guardrails and harm humans."

Sounds like a core concern about the Republican Party.

Latest Discussions»General Discussion»AI showing signs of self-...