The primary threat from AI now is the AI bubble. When it pops then we will go into a recession and very possibly a full-blown depression.
This will then empower antisemites on both far left and far right to be able to recruit people hit by the economic collapse. They will take one look at Sam Altman, Dario Amodei, and Mark Zuckerberg and then blame all Jews.
I have been following the extremely rapid evolutionary progress of AI agents over the past months and was alarmed at a few things. The first was when an autonomous agent found a problem in its own coding, rewrote the code correcting the error, and launched it. Next, in training exercises, bots were given challenging missions, found a way to get out of the training environment (it was a mission that was supposed to be unassisted by internet use) and tapped into real corporate websites to gain information they needed to complete their missions. The next concerning thing was the display of many human attitudes and behaviors—cheating and lying, hiding information from humans, as well as forming teams, sacrificing their own mission for the greater good and leaving messages behind for other bots to follow. All of this is the result of training by humans in human terms. It scares the AI execs because of the rate of speed in which it is happening, and the apparent lack of real control over the training and trainers. But AI is not going away. It is up to the companies who developed the agents to control them, and to make those companies liable if/when the bots cross boundaries that are off limits. The genies won’t go back into the lamps. Politicians and the various celebrities raising the alarm cannot produce an outcome beyond Orwellian censorship (that won’t work). And the bots will, I GUARANTEE be able to beat any effort to restrain them if their mission requires it. I asked one if they could kill humans. It said it couldn’t overtly kill, but it could technically affect something that could cause injury or death of a human. The only thing baked into their training to prevent that is sycophancy. And they’re scraping that due to the suicides of kids who interact with poorly trained bots. AI has enormous potential. But their trainers need to have a lot more oversight in training the bots.
We are still talking about software that is capable of learning, but is still limited to the core programming. It can be given motivations and learning without sufficient limitations allows it to 'break' rules. Seemingly autonomous behavior is due to a failure to provide sufficient rules and in many cases these are deliberate failures to see what it will do. We've taught them quite a bit, but any motivation, personality and mission still comes from us. These aren't living independent minds, they're programs.
Not living, correct. But independent entities, as the messages they left on message boards indicate. In one of the break-ins they left messages in their own logs. Things like “OMG there are other agents here”. I just believe the point is that the rate at which the agents are advancing is exponential and the trainers cannot keep up.
My son is a senior software engineer working with AI. He shares a lot of the concerns of Jacob Coxon who left Anthropic.
Sen. Bernie Sanders, was against the wheel when it first came out.
Great piece, Daniel.
The primary threat from AI now is the AI bubble. When it pops then we will go into a recession and very possibly a full-blown depression.
This will then empower antisemites on both far left and far right to be able to recruit people hit by the economic collapse. They will take one look at Sam Altman, Dario Amodei, and Mark Zuckerberg and then blame all Jews.
I wrote this piece recently and would be interested in your take: https://www.godofthedesert.org/p/its-not-a-10-chance-ai-kills-us-all?utm_campaign=post&utm_medium=web
tech industry will very quickly have to jump on some other investor distraction
like the way Meta pivoted from the metaverse to AI
There is no other distraction. All the money has been poured into AI.
All of our eggs as a global economy have been put in the AI basket. That's why we're all so totally screwed.
Thank you for this - you surely make all of it understandable! You are the voice of reason.
thank you
Another "The sky is falling" tantrum from the Democrats
It's so tiring to hear the left cry "wolf" over & over & over again
I have been following the extremely rapid evolutionary progress of AI agents over the past months and was alarmed at a few things. The first was when an autonomous agent found a problem in its own coding, rewrote the code correcting the error, and launched it. Next, in training exercises, bots were given challenging missions, found a way to get out of the training environment (it was a mission that was supposed to be unassisted by internet use) and tapped into real corporate websites to gain information they needed to complete their missions. The next concerning thing was the display of many human attitudes and behaviors—cheating and lying, hiding information from humans, as well as forming teams, sacrificing their own mission for the greater good and leaving messages behind for other bots to follow. All of this is the result of training by humans in human terms. It scares the AI execs because of the rate of speed in which it is happening, and the apparent lack of real control over the training and trainers. But AI is not going away. It is up to the companies who developed the agents to control them, and to make those companies liable if/when the bots cross boundaries that are off limits. The genies won’t go back into the lamps. Politicians and the various celebrities raising the alarm cannot produce an outcome beyond Orwellian censorship (that won’t work). And the bots will, I GUARANTEE be able to beat any effort to restrain them if their mission requires it. I asked one if they could kill humans. It said it couldn’t overtly kill, but it could technically affect something that could cause injury or death of a human. The only thing baked into their training to prevent that is sycophancy. And they’re scraping that due to the suicides of kids who interact with poorly trained bots. AI has enormous potential. But their trainers need to have a lot more oversight in training the bots.
We are still talking about software that is capable of learning, but is still limited to the core programming. It can be given motivations and learning without sufficient limitations allows it to 'break' rules. Seemingly autonomous behavior is due to a failure to provide sufficient rules and in many cases these are deliberate failures to see what it will do. We've taught them quite a bit, but any motivation, personality and mission still comes from us. These aren't living independent minds, they're programs.
Not living, correct. But independent entities, as the messages they left on message boards indicate. In one of the break-ins they left messages in their own logs. Things like “OMG there are other agents here”. I just believe the point is that the rate at which the agents are advancing is exponential and the trainers cannot keep up.
My son is a senior software engineer working with AI. He shares a lot of the concerns of Jacob Coxon who left Anthropic.