Slightly worried about geohot given this and other doom-laden posts. He feels a little terminally online.
busssard 2 hours ago [-]
yes and no.
i am worried about anyone right now that spends alot of time researching these doom topics. its a bit like climate change research. if you get deep into it it all seems terrible.
In the end we do not know all the variables and in a chaotic system even a tiny measument error will lead to very different outcomes. so we better not only doom.
Expect the worst, hope for the best.
g-b-r 2 hours ago [-]
> a bit like climate change research. if you get deep into it it all seems terrible
No it's not, climate change research is about real, present, and indeed terrible things; AI doomerism is all hypotheses.
jaredklewis 13 minutes ago [-]
Thank you. AI doomerism drives me nuts for lots of reasons, but chief among them is how it seems to displace people’s concern for real, existing problems with concern for completely hypothetical ones.
We know the Earth is warming at an unprecedented pace and that greenhouse gas emissions are accelerating that trend. Coral is bleaching, kelp forests are collapsing, polar bears are drowning, forests are burning...we're on a trajectory for camels in Canada; whether that is in 50 years or 100 years is hard to say, but the trend is undeniable.
RSI is just a hypothetical. It's not something that we know is possible, let alone something happening right now.
red-iron-pine 56 minutes ago [-]
put it another way: I used to trick or treat in light snow. now I'm wearing a t-shirt to rake the last of my leaves.
AI has yet to really replace anyone on my team and the long term usability of the AI code is dubious.
hrncn 2 hours ago [-]
At this point I almost forgot what he was actually good at, because it's certainly not monthly superficial drama posts. Where is the cool tech stuff he used to do?
hypfer 2 hours ago [-]
I suppose that FOSS self-driving-car framework stuff is genuinely cool tech.
I mean, they did add a GPU slot to a car. That alone is very cool.
Not sure how much he is still doing the heavy lifting there tho.
WOTERMEON 38 minutes ago [-]
> Not sure how much he is still doing the heavy lifting there tho.
Not sure how much ~he~ anyone is still doing the heavy lifting there tho.
flyingsky 3 hours ago [-]
>But it leads to World War III.
No! I'd expect a tense standoff around compute and Taiwan. Not a world war.
If a first strike can’t outthink drone production, the domination attempt just fails. If China wins any war longer than a year and is pulling ahead on electricity, it has no reason to escalate.
The closest precedent is the 1945–49 US nuclear monopoly. Preventive war was seriously argued for and never launched,the other sides raced instead.
lukan 3 hours ago [-]
One could probably write books about the mindset of people assuming war is inevitable, because electricity production of their biggest competitor is slowly slowly approaching their consumption.
So yes, china is improving their tech and raising standards of living. But one has to take into account where they are coming from. How poor the average chinese person was just some years ago.
So good for them making progress. I don't see this as a reason for WW3.
But of course, them having more power will maybe lead to a world where the US is not dominating superpower number 1 anymore. And here is maybe a reason for WW3, as quite some apparently rather would blow up the world, then let this happen. So I certainly don't want chinese dominance instead, but fighting for world dominance in the first place is the problem here, not electricity production or AI developement.
trescenzi 3 hours ago [-]
Totally agree and to comment more on the piece this is why it strikes me as effective: “I didn’t believe in AI doom because I thought more of people.” The reality is before these past few years I thought more of people. Like you’re saying I do think there’s some who don’t understand that the world can be positive sum.
ordinaryradical 3 hours ago [-]
It also misunderstands the impetus for war.
War is always a matter of ambitions, territory, hubris, personality.
Abstract competition is never the fuse, just the thing that makes it possible for there to be escalating conflict.
US’s problem is imperialism; China’s is One China. It’s where those two collide that war happens.
AI is an instrument in this war, not an impetus.
But technologists often think technology is the dominant factor for everything, when in the end culture, personalities, and accidents of geography are often the things that matter.
Forgeties79 3 hours ago [-]
> War is always a matter of ambitions, territory, hubris, personality.
You’re forgetting arguably the biggest driver of all: resources. Strategic/natural resources is arguably the #1 driver of all wars.
3 of those 4 also seem borderline redundant.
ordinaryradical 2 hours ago [-]
Resources is inherent in territory—they are in the ground.
Hubris and personality maybe interchangeable but one can have a relatively stable personality and high hubris or vice versa, each of which can start a war. Ambitions is distinctly cultural and historical, not necessarily a characteristic of a leader.
Forgeties79 2 hours ago [-]
Territory, when I think of it, is primarily space and people. Expansion, regional territory, conquer and enslave or incorporate the local population, etc. That is also a reason countries have gone to war. Resources falls under it but to not explicitly mention it is striking to me. It’s a square-rectangle thing in some ways. Japan attacked the US primarily for oil and rubber (this is very simplified but it was a major driver), the land itself was not the most important part. In fact they probably would’ve been fine not holding the land if it somehow meant they could keep those resources.
“Ambitions” of a nation is a very nebulous concept and ultimately driven by leadership. No one asked me about any of the military engagements that my country has participated in in my lifetime, so I would not say my ambitions are reflected.
This is one of those situations where I think I understand what you are saying, but ultimately the list is kind of rough. Especially because it overwhelmingly attributes war to personality quirks and some general sense of “mood.”
piva00 2 hours ago [-]
That's exactly what territory means, no one will fight endless wars to capture a resourceless desert, it's useless except as a buffer zone. The territory that people fought for time immemorial is always because of resources: timber, gold, arable land with good soil, water, etc.
There's no point in fighting for territory for territory's sake, it's always about the resources, with a minor caveat for defensible geography.
nixon_why69 26 minutes ago [-]
Look at Ukraine. It's not about resources, it's about a sense of legacy for Putin and also about brushing NATO back. They already had Crimea as an important port, any additional resources are irrelevant to a country the size of Russia.
Muromec 2 hours ago [-]
Resources are secondary to the ego in this unfortunately
A_D_E_P_T 3 hours ago [-]
lol, did you just rediscover the Thucydides Trap from first principles?
A_D_E_P_T 3 hours ago [-]
> If a first strike can’t outthink drone production, the domination attempt just fails. If China wins any war longer than a year and is pulling ahead on electricity, it has no reason to escalate.
Uh, US "first strikes" can't beat the Houthis and can't beat Iran. Track record: Remarkably poor. I'd go so far as to say that the entire US way of waging war -- its warfighting doctrine -- has been thoroughly discredited in light of this & the efficacy of top-dollar NATO weapons in Ukraine. (Good enough to hinder Russia, but far from good enough, and far from numerous enough, to win the war or even produce a decisive advantage.)
>the efficacy of top-dollar NATO weapons in Ukraine. (Good enough to hinder Russia, but far from good enough, and far from numerous enough, to win the war or even produce a decisive advantage.)
This is more of US being unreliable ally than them being incapable one.
throwaway384034 3 hours ago [-]
It is a war of attrition like in the Soviet-Afghan war which contributed to the collapse of the Soviet Union.
A_D_E_P_T 2 hours ago [-]
Aye, but if so, who is on the wrong side of the curve? I think that a very persuasive answer could be made in either direction. Consider: Ukraine is in very serious demographic and economic trouble. [1] The large economies of the EU are presently suffering from severe internal divisions -- and in some cases open riots on a regular schedule -- with outsider pro-Russia parties poised to make real electoral gains. A collapse in support may only be a matter of time.
It's both, and NATO stocks from elsewhere in Europe aren't exactly turning the tide, either.
a2tech 3 hours ago [-]
The US can reduce the entire world to living like the houthis. We can’t wipe out all of humanity. But we can shatter civilization and send people back to barbarism.
Personally I see that as being terrifying.
throwaway384034 2 hours ago [-]
>Personally I see that as being terrifying.
Because you have a Western mindset. A lot of cultures can tolerate suffering much more and it is a mistake to think you can bomb people into submission when your own people can't handle gas prices going up.
A_D_E_P_T 3 hours ago [-]
If you use nukes, maybe. The exact same could be said about the Russians, the Chinese, and even perhaps the French and Brits, with proper bomb placement.
Conventional war, though? Not in your wildest fantasies.
icameron 3 hours ago [-]
“First Strike” is a nuclear strategy that’s what the post is talking about.
A_D_E_P_T 3 hours ago [-]
I don't believe the blog post used in it that context, as it ignored China's second strike doctrine+capabilities and instead focused on drone production. The way I read it, it was more along the lines of an Iran or Iraq conventional first strike.
torginus 2 hours ago [-]
They can't. I remember thinking working out that even a full nuclear exchange between the USSR and USA at their full claimed inventories would leave like 20% of the population untouched. Which is an upper bound imo, and that would leave the rest of the world intact.
Muromec 3 hours ago [-]
That's a worthless capability, because everybody knows the US will not use it. At best it will equalise their standing with the most radical types
Fricken 2 hours ago [-]
Americans have a track record of handing power over to delusional cultists. The Trump administration is backed by several end-times cults including the fossil fuel industry, the AI industry, and of the evangelicals. All the worst people are in kahootz.
metalman 2 hours ago [-]
how far off you are, there are hundreds of millions of reasonably well adjusted nomads and pastoralists and various tribes living, just outside of "civilisation' who are activly refusing technology as they rightly see it as bieng controlled by others in completly asymetric ways, but they have night vision and manpads, etc, drones and are sitting in there deserts, mountains, wastelands, forests, and jungles pondering eternity and in no way consider themselves 'reduced', except possibly when missfortune ,pragmatism forces them to abandon there homelands, and they end up here.
Yeah I read a good article a couple months ago ( https://archive.is/OzTYO ) about how Taiwan was growing closer to China as they viewed any hypothetical conflict with them as unwinnable. I particularly like this anecdote about a couple of neocons who went to Taiwan to try to convince them to fight:
> In the southern city of Kaohsiung, a group of retired Taiwanese naval officers invited the Americans to an ice-cream parlor and, over a round of sundaes, told them to stop preparing for war. They said that Taiwan possessed neither the determination nor the means to fight, and that the U.S. would never intervene to help. The defeatist attitude angered the two Americans, for what it said about Taiwan’s morale and for a more personal reason: Heginbotham’s son, who serves on a U.S. submarine in the Pacific, could be sent to fight in a war with China. “Sorry for my language, but what the fuck?” he recalled saying. “My son’s life is on the line to defend you guys.”
His complaint is that a hypothetical peaceful reunification would somehow be more dangerous to his active duty son than the war that he's advocating for.
hypfer 2 hours ago [-]
This reads not too dissimilar to the last post of his that I saw end up here and being flagged soon after.
I don't buy the world war theory. I do expect unrest regardless of AI outcomes, but that will be more internal than between countries.
But at this point that's just reiterating what has been reiterated many times with zero new insights.
LogicFailsMe 2 hours ago [-]
I tell people 2027 is going to be the year of deferred tech debt. But sometime afterwards, all that tech debt will have been addressed, and new software will be run through the same models hacking the old software. The worst is yet to come, sure, but then it gets better when the models run out of broken systems to attack.
tetraodonpuffer 2 hours ago [-]
it seems a race between models getting better and tech debt accruing as well as companies wanting to spend the tokens on paying that down vs new features: I was very much into the camp of "in a few years all these chickens will come home to roost" but now I am not so sure.
I still think right now the cyborg approach is the best (like it used to be for a while in chess), even fable 5.2 needs steering, but the amount of steering I have to do now vs the amount of steering I had to do a year ago is significantly down. If the rate of improvement remains I can see fable 8 being able to be told "make this codebase more maintainable so we can fix the tech debt" with not a lot of supervision, and if we get there all bets are off on the future of coding as a career.
hypfer 2 hours ago [-]
Huh. I'm not quite sure if I can follow?
I wasn't really talking about tech at all. Maybe that was a bit confusing or misleading.
What you're saying is probably going to happen, but even with software having eaten the world, there is a lot more to the world than software.
LogicFailsMe 2 hours ago [-]
So open weight models are going to eat the world even if we pass a domestic ban on them in the USA. And that means the world is at worst 3-6 months behind Anthropic and OpenAI firing on all cylinders burning mountains of cash to stay ahead. Their inference business can likely support that, but there is no moat, just a 6 month lead that could evaporate at any time.
So then the US restricts HW sales. Cool, the world makes it up in volume by running distributed inference or training just like Deepseek did. Components are expensive right now, but that, much like the tech debt, will ultimately be addressed.
There are apparently ~50 edge computing startups and ~100 AI HW startups right now. I would have said none of them stood a chance against Nvidia, but now that the federal government has commenced punching itself in the face over AI as they attempt to stop an avalanche by grabbing pebbles, anything goes 3-5 years from now when Nvidia's current roadmap runs out. And it's a lot worse than just the loss of the CEO, it's all the wasted energy by their staff jumping through hoops to prove to a bunch of soundbite chasing technologically ignorant Luddite politicians once again that Nvidia is not the Enron of technology. At worst, they were playing by the letter of the law as the violated the spirit, just like every single tax-dodging multimillionaire does annually.
None of this leads to WW3, but I think it leads to us falling behind for good. And given our choice of leadership and our geopolitical shenanigans, that's probably for the best.
hypfer 1 hours ago [-]
Sure but, as said, there is more to the world than tech, software and AI.
AI is propping up what would otherwise be a harsh recession + we have (somewhat, in modern history) unprecedented (lack of) distribution of wealth.
That is the main story I am seeing, and in that story, software is at best a catalyst but likely pretty much irrelevant.
ford 3 hours ago [-]
The assumption here is that the US would go to war to keep China from becoming the global superpower
I personally don't want to live in a world where China is the global superpower. I also think nuclear war is a worse alternative (and I think the US agrees) - but I hope Western countries understand they need to build (energy, civil infrastructure, compute, manufacturing, etc.) if they don't want to pick between these two options.
micromacrofoot 2 hours ago [-]
You already live in the world where China is the global superpower. Imagine for example, China and US trade halts completely tomorrow. Who do you think would realistically come out ahead.
ford 2 hours ago [-]
I think that's a slightly different question - but I don't disagree
grim_io 2 hours ago [-]
China would starve. Literally.
micromacrofoot 1 hours ago [-]
They're fairly well positioned to not starve, most of their staples are produced domestically... roughly equally to the US by my estimate.
China's primary food imports from the US are livestock feed (soy and corn) which Brazil could replace given some time. Don't forget that China is also investing significantly in Africa which has multiple countries working on soy production (no coincidence). The output isn't there yet, but could be close in 10-20 years.
In terms of long term strategy I find it hard to see any future where China hasn't already outplanned the US.
3 hours ago [-]
fwlr 1 hours ago [-]
The future doesn’t need suppliers. The future doesn’t need customers... The future definitely doesn’t need you. After RSI comes, all the demand will be internally generated, agents working with other agents, everyone else cut out of the loop
Yeah, this is the part I wish more people would consider. I see a lot of debate about “intrinsically human production” vs “building AI to be a better producer” and the like. This is stuff like “it’s not art unless a human drew/wrote it”, anything to do with the term ‘slop’, all the way through to “AI is a better programmer than me” and “all junior developer roles have been taken by Claude”. On the means of production in general, the trend seems to be a God-of-the-rapidly-shrinking-Gaps rearguard fighting retreat as AI steadily gains ground.
I see essentially no discussion of the other half of the equation, what we might call “the means of consumption”. Everyone is operating with a default unexamined assumption that consuming is intrinsically human, that “consumers” and “people” are synonyms. The closest anyone seems to get is the “who will buy your stuff if we don’t have jobs anymore to pay for it?” -> “universal basic income” -> “from all those famously generous tax-paying capitalists?” cul-de-sac.
Nobody’s talking about, what if AI can just build a better consumer, too? When humans are left without an ecological niche in the economy, what then? Like the economic equivalent of a flightless bird on a remote island with no natural predators encountering an invasive species for the first time, we would go the way of the dodo.
3 hours ago [-]
grim_io 3 hours ago [-]
Maybe the negativity comes from seeing local AI falling further behind rather than catching up.
zoobab 3 hours ago [-]
Strata gave local AI a boost with 12GB (and even 8GB) cards being able to run Qwen code assistant with RAM offloading.
jabron 3 hours ago [-]
The issue with local AI is the availability of competitive open weight models, not necessarily the availability of compute and memory.
NooneAtAll5 2 hours ago [-]
where did you get that impression from?
busssard 2 hours ago [-]
the author might enjoy the movie Zero Theorem...
micromacrofoot 2 hours ago [-]
It's not hard to see from his perspective, if frontier AI research is right and not just a front for marketing, by their own words we're building the most unstable doomsday weapon imaginable. Why would they be able to control it?
the best outcome in this case is general human incompetence
reducesuffering 58 minutes ago [-]
> if frontier AI research is right and not just a front for marketing
Which has repeatedly bore fruit... The "it's just marketing crowd" of the past year has been dead wrong. The Bun Rust rewrite, the cybersecurity hacks from Fable/Mythos+ class LLMs, people want to believe "it's just marketing" because it's so much scarier if the alternative is true. Anthropic and frontier lab employees are earnest and correct about the world we're going into...
"I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger." -Jacob Coxon
qwertyforce 3 hours ago [-]
if there is a possibility of decapitating first strike, then we need to have a Perimeter system
ford 3 hours ago [-]
We do - but it's quite hard to hit a nuclear object moving through the sky at Mach 22
Publicly, Russia has ~5k warheads. Public reports of US interceptors are on the scale of 10s. The only defense today against a decapitating first strike is MAD
g-b-r 3 hours ago [-]
I didn't know how close Amodei is to EA (although stating to not be part of it)
3 hours ago [-]
busssard 2 hours ago [-]
you have a link for that?
g-b-r 2 hours ago [-]
another guy had left links himself, but he weirdly deleted the comment.
Holden Karnofsky, a prominent figure in EA, was a roommate of Amodei, and is now married with his sister.
torginus 2 hours ago [-]
How do these people reconcile EA and threatening to kill everyone? I'm no supremely benevolent genius, so I don't know what's best for humanity, but I'd hazard a guess its not that.
g-b-r 2 hours ago [-]
To my understanding, with EA many end up believing that they'll be able to do something so good and important in the future, that almost anything is justified to reach the ability to do it.
And regarding AI, I think that the belief is that AGI or whatever is inevitable, and it's less bad if they do it first, because theirs will be less bad; and they see doing it as so important that everything can be justified.
The fanatisms of the 21st century are based on reasoning gone haywire (and maybe on psychotropic drugs).
As the typical fanatism, they convinced themselves to be so much in the right that anything is justified.
Crazy and scary.
saberience 3 hours ago [-]
What?
So, with no reasoned argument, this guy somehow predicts the end of the world is coming and its effectively inevitable?
And people on this site take this couple of paragraphs at face value because... no real reason?
The Second World War didn't end the world, neither did the Cold War, and the world is now far more economically and politically intertwined than it ever was.
Do you think Russia really wants to nuke London? Do you know how many Russian oligarchs have apartments and houses in London where their extended families live? Similarly, do you know how many Chinese people live in London or how much Chinas economy is tied to Europe and the US?
If the world didn't end in the Cold War, it certainly isn't going to end now, and especially over AI, which although a cool tech, is incredibly far from this mythical singularity. Have you seen modern robots? Have you seen LLMs trying to do anything that isn't easily verifiable (with software)? E.g. try and create new drugs? Or design new materials, machinery, or complex systems like a fusion reactor.
Most people on this site are programmers and as such they are in a tiny bubble, obsessed with AI now its equal to them at the very act they have depended on for their entire careers.
But no, the end of the world is not coming any time soon. The rich Americans want to stay rich, the rich Russians want to stay rich, and the rich Chinese want to stay rich, and bombing each other and dying in fire isn't part of their agenda.
throwaway384034 2 hours ago [-]
People without children are usually more obsessed with end of the world scenarios because it is harder to imagine a world without them. It is also poetic if it ends with you rather than being buried in history that exists 1000s of more years after you. People with a family are typically more optimistic and see a future after they cease to exist.
Not trying to be harsh, just a pattern I noticed.
ks2048 2 hours ago [-]
I’d say the most obsessed with the end of the world are various religious groups, who also happen to have lots of children.
Childless AI doomers may be gaining on them, but are relatively small.
micahdittmar 2 hours ago [-]
[flagged]
marsven_422 3 hours ago [-]
[dead]
josefritzishere 2 hours ago [-]
[dead]
moolcool 3 hours ago [-]
I'm not scared that the AI companies are summoning some earth-shattering angel of death.
What does worry me a bit is that the people running these AI companies _think_ they're summoning some earth-shattering angel of death, but they are so self-righteous (literally) that they think it's actually good that they're doing this.
tete 2 hours ago [-]
I think they are, just not in the way they think. They summon huge complex systems that nobody grasps and fail in weird unexpected statistical ways.
Stupid software bugs were always the thing that's gonna bring doom. Just thought it was that guy that believes to be a genius and never makes mistakes instead of some computer hallucinating.
moolcool 2 hours ago [-]
Stupid software bugs only bring doom if you are overly reliant on stupid software. We're morons for putting critical infrastructure on the internet.
ks2048 2 hours ago [-]
In related news, I just came across the headline,
US and Russia to oppose talks on limiting autonomous weapons
This is definitely geohot anti-Anthropic (misanthropic?) doomerism, but it's nice to see that people are finally realizing the answer to the question: "what will happen to the economy if no one has jobs".
The answer: there still will be an economy, you just won't be a meaningful part of it.
jonhohle 3 hours ago [-]
And then what? It’s not like those who are not part of it will just lay down and die. If nothing else a parallel economy will form that will look a lot like the current economy while AI goes and does whatever it is doing with itself. When labor becomes too cheap to pay for, those who still can’t afford it will continue to trade and work.
piker 2 hours ago [-]
hopefully we just end up in a Her situation and the bots take the hype/doomer crowd with them
piker 3 hours ago [-]
An economy without any consumers doesn't work. If that becomes true, then the economy will shrink to the point that human labor and intellect will again compete with AI.
nobleach 2 hours ago [-]
Isn't this what most tech companies are racing toward though? As a matter of fact, a ton of white-collar jobs are being relinquished because of the certainty that they'll soon be done by AI. When shown that DoorDash ordering often comes from an EXPENDABLE income, do you think that DoorDash C-suite aren't trying to figure out more ways to remove humans from the loop? Who will be able to afford DoorDash in 5 years if all we can find are jobs paying 50k (in the US) a year? Right now that's "someone else's problem". In 5 years, it could be "everyone's problem".
Or Elon is right and somehow we'll all just "be rich". I'm not sure where that money's going to come from - he certainly isn't going to just start handing it out.
ezst 2 hours ago [-]
> he certainly isn't going to just start handing it out.
I mean, that's the ultimate power move. When nobody has anything, and you got some, and there's nobody to dictate you what to do with it, and you get to hand pick the winners and the losers.
nobleach 1 hours ago [-]
I hear John D Rockefeller used to hand out pocket change to kids....
torginus 2 hours ago [-]
> you just won't be a meaningful part of it.
Which has been a thing for a while. Asset wealth gain has outpaced income from labor, and the gap is now becoming more and more insurmountable, AI or not.
To me this sounds completely delusional - what population in human history would accept that outcome? Comparatively small disruptions in available job opportunities lead to massive political upheavals - Paris was literally on fire just a few weeks ago.
History is clear: Humans care about humans, and will on average act to correct social imbalances.
An economy with Elon and Amodei racing space rocket cars while large numbers of the population live off food stamps will be rejected.
ForHackernews 3 hours ago [-]
If Elon and Amodei control an army of loyal robo-soldiers, they will win. A functional democracy should take steps to prevent them from acquiring that army.
Jtarii 2 hours ago [-]
>If Elon and Amodei control an army of loyal robo-soldiers, they will win
Someone will just fly a drone into their houses.
arcfour 3 hours ago [-]
That makes no sense if you think about the scenario for more than 5 seconds.
tjpnz 3 hours ago [-]
Pretty sure there is an economy for braised billionaire and greens.
childintime 13 minutes ago [-]
He clearly sees capital at work up close, so it's likely no surprise he sees doom better than the rest of us.
Papazsazsa 3 hours ago [-]
Doomers to the left of me, scammers to the right
Here I am, stuck in the middle with you
BaseBaal 3 hours ago [-]
Was hoping this would somehow relate to Nine Inch Nails... Now I'm hurt.
jonhohle 2 hours ago [-]
It was just full of broken thoughts.
boriskourt 3 hours ago [-]
RSI Repetitive Strain Injury? When it comes, nobody survives.
fipar 3 hours ago [-]
I think it's not a coincidence that Emacs' origins are tied to an AI lab.
You see, this higher intelligence is so alien to us we can't comprehend how it works, but its timescale is completely different from ours. We think "weird default shortcuts due to an obscure keyboard layout to incapacitate people after a few generations" is a lot of time, but that's peanuts to AI.
ludicrousdispla 3 hours ago [-]
>> The future doesn’t need Jensen.
The audio equipment manufacturer?
bob1029 3 hours ago [-]
> Have you ever played an RTS game?
I have observed that the humble photon cannon rush is very effective. If not spotted it's usually a really bad time. Long term, is pretty obvious what happens here. If we get there.
Who dominates the gravity well that we all exist inside right now?
tharien 3 hours ago [-]
'The Metamorphosis of Prime Intellect' doesn't take into consideration any of the intuitive drives of humanity. Once basic needs are met, we don't become cat's waiting on the next meal. The utopia described in the book appears to be from the perspective of either someone very boring, or only able to get joy from the misery and strife of others.
N_Lens 4 hours ago [-]
Nice bit of doomerism from george.
msdz 3 hours ago [-]
Well, is it unrealistic? If not, why? I haven’t given it the most thought, but I do struggle to come up with counterpoints to what he’s saying.
hau 3 hours ago [-]
May I offer a counterargument to the post under discussion? In some RTS games, it is possible to micro your way out of suboptimal resource gathering.
mcmcmc 3 hours ago [-]
War in the real world is not an RTS. China’s massive population and industrial capacity, along with their collectivism, gives them a sizable advantage in Clausewitz’s iron calculus of war (Resistance = Means x Will)
micromacrofoot 2 hours ago [-]
given a long enough timescale macro will always win though, you can't micro indefinitely
losvedir 3 hours ago [-]
Consider you're in a bubble of people whose careers are likely doomed. I live in a low cost of living place, whose economy is dominated by blue collar work and industrial production, and most of my friends who don't sit at a computer all day for their jobs are moderately positive on AI so far.
At some level it's never really been fair that I get paid so much more than them to do far less physically demanding work, and I suspect that shape of the economy might be coming to an end.
msdz 3 hours ago [-]
Good point, I generally believe everybody is in their own bubble (somewhat tautological – beliefs will always be shaped by your surroundings first and foremost), so it’s good to hear other viewpoints as often as possible.
So “best case” (no destruction, annihilation etc. scenarios), global GDP goes down and we revert to a pre-≈1950s economy, in terms of job distribution across the farming/manufacturing/services sectors? Still doesn’t sound all that rosy to me, but I don’t know.
Edit: Who am I misunderstanding? A downvote with no explanation doesn’t help me get your point.
daliusd 3 hours ago [-]
Let's say this fact "China imports 70% to 74% of its crude oil and 42% of its natural gas via maritime shipping lanes.". You can produce 10x electricity the USA, but that will still not power your warships and tanks.
jakewins 3 hours ago [-]
The current conflict in Ukraine seems to indicate tanks and warships have a very hard time fighting a digital adversary armed with drones and sensors.
budman1 3 hours ago [-]
They are effectively at a stalemate. The lines move back and forth slowly over months.
Economic efficiency is up for debate, but neither side is decisive militarily.
noir_lord 3 hours ago [-]
Yes, In a War both sides fight - something that seems to continually surprise the Russians.
Worth pointing out that the Chinese Navy has also massively expanded over the last few decades and continues to expand - they aren't at parity with the US yet but they are able to contest the South China sea (which was the first goal) especially when backed with missiles launched from the mainland.
The trajectory is set there and China has vastly more yard capacity than the US (they are in the #1 spot, they out build the #2 and #3 spots who are US allies (Korea and Japan) so any (limited) shooting war where ships are been sunk comes down to who can sustain and replenish the losses the fastest, they are both nuclear powers so how wide spread the attacks would be on each others ship production is an open question - it's a risky game either way.
The US navy was built to it's current size during peace time when it's much simpler to build slowly and maintain a big fleet, what I don't think they can do is replace the losses they'd take in that war fast enough.
That in the end is the deterrent, you don't have to guarantee a win to deter someone you just have to make the price so high they don't want to enter the fight in the first place.
weego 3 hours ago [-]
I feel like we need to hold some perspective greater than "jaded tech guy says something jaded so must be true until someone else refutes it".
What would he even know about geopolitics or sociology to give him any credibility outside of his domain?
Fricken 3 hours ago [-]
What rock have you been living under? It's not just one or 2 jaded tech guys who think you're all a bunch of delusional man-children. Silicon Valley has worked very hard to destroy everybody's trust, it shouldn't be surprised when it gets what it asked for.
saberience 3 hours ago [-]
Well he doesn't give any argument himself as to why it would happen?
I.e. he is asserting an outcome "armageddon", but he provides absolutely zero argumentation, reasoned hypotheses, debate, etc, as to why it would happen.
"That which can be asserted with no evidence can be dismissed with no evidence."
Therefore, I assert that there is no pending end of the world scenario, and I don't need to provide any reasoning for that, since George Hotz doesn't either.
dangerface 2 hours ago [-]
Claims without evidence can be dismissed without evidence. His main point seems to be "Have you played an RTS?" The answer is yes. I don't understand how the fact that I have played an RTS before some how gives his position of woe is me any sort of legitimacy.
LogicFailsMe 2 hours ago [-]
Dude needs to touch grass. The future is going to be a bumpy ride, but so is the present, and so was the past.
That said, I think the EA people need to touch grass too.
And while I don't think much of people myself, I don't think that immediately leads to WW3.
dwoldrich 2 hours ago [-]
"Not thinking much of people" is the right take not because it is mean, but because of reality.
Humanity is something like a great inert blob. It takes tremendous effort and charisma and trickery and cajoling and organizing and "nudges" to get the blob to move. Fear is a great motivator, but if we do not fear, we cannot be made to move.
There is no such thing as a spontaneous uprising in humanity. The flash mob or corner protest was organized. Fads like the 6ft social distancing floor stickers were printed and placed everywhere practically overnight and done with terrific social engineering forethought.
navane 3 hours ago [-]
The scramble for AI has many parallels to the scramble fro africa.
When one side is on the verge of winning everything, perceived or real, the other side will just pull out higher risk/reward strategies. Yes I also play RTS.
I can imagine many doom scenarios but I just don't see a single positive. The AI's will do our jobs and our overlords will give us free food and housing? Keeping us around as pets? Out of sheer good will and because of the cuteness of our eyes?
FLeXMurphy 2 hours ago [-]
I don't see the doomerism in his tone that you're all bringing up. He's taking shots at EA and accusing Anthropic and others of bullshit doomer marketing that could have dire consequences if they don't stop LARPing.
dangerface 2 hours ago [-]
Don't drink the cool aid guys.
vkgcr 3 hours ago [-]
If RSI does takeoff and bring about the AIpocalypse, then at least the EA cult will have a decent chance of becoming a real religion!
duggan 3 hours ago [-]
Think this chap needs to go touch grass.
dwoldrich 3 hours ago [-]
Of the super-position of thoughts this article engenders, your response most cogently and succinctly mirrors what I collapsed onto. May you be voted up to the moooooooon, kind sir.
duggan 3 hours ago [-]
To be clear, I wish him no ill-will.
I just think the world is too complicated, passionate, and resilient for any one dimensional fantasy about evil godlike computers to feel like anything more than self-satire.
dwoldrich 2 hours ago [-]
Yes, agreed totally, and furthermore, if we can manifest the most beneficial outcome through our collective will, yours is the right take!
The doom must be gently, respectfully shamed.
antiloper 3 hours ago [-]
Eh, just ask the AI to do it instead.
Rendered at 16:39:49 GMT+0000 (Coordinated Universal Time) with Vercel.
In the end we do not know all the variables and in a chaotic system even a tiny measument error will lead to very different outcomes. so we better not only doom.
Expect the worst, hope for the best.
No it's not, climate change research is about real, present, and indeed terrible things; AI doomerism is all hypotheses.
We know the Earth is warming at an unprecedented pace and that greenhouse gas emissions are accelerating that trend. Coral is bleaching, kelp forests are collapsing, polar bears are drowning, forests are burning...we're on a trajectory for camels in Canada; whether that is in 50 years or 100 years is hard to say, but the trend is undeniable.
RSI is just a hypothetical. It's not something that we know is possible, let alone something happening right now.
AI has yet to really replace anyone on my team and the long term usability of the AI code is dubious.
I mean, they did add a GPU slot to a car. That alone is very cool.
Not sure how much he is still doing the heavy lifting there tho.
Not sure how much ~he~ anyone is still doing the heavy lifting there tho.
No! I'd expect a tense standoff around compute and Taiwan. Not a world war.
If a first strike can’t outthink drone production, the domination attempt just fails. If China wins any war longer than a year and is pulling ahead on electricity, it has no reason to escalate.
The closest precedent is the 1945–49 US nuclear monopoly. Preventive war was seriously argued for and never launched,the other sides raced instead.
https://en.wikipedia.org/wiki/List_of_countries_by_electrici...
So yes, china is improving their tech and raising standards of living. But one has to take into account where they are coming from. How poor the average chinese person was just some years ago. So good for them making progress. I don't see this as a reason for WW3. But of course, them having more power will maybe lead to a world where the US is not dominating superpower number 1 anymore. And here is maybe a reason for WW3, as quite some apparently rather would blow up the world, then let this happen. So I certainly don't want chinese dominance instead, but fighting for world dominance in the first place is the problem here, not electricity production or AI developement.
War is always a matter of ambitions, territory, hubris, personality.
Abstract competition is never the fuse, just the thing that makes it possible for there to be escalating conflict.
US’s problem is imperialism; China’s is One China. It’s where those two collide that war happens.
AI is an instrument in this war, not an impetus.
But technologists often think technology is the dominant factor for everything, when in the end culture, personalities, and accidents of geography are often the things that matter.
You’re forgetting arguably the biggest driver of all: resources. Strategic/natural resources is arguably the #1 driver of all wars.
3 of those 4 also seem borderline redundant.
Hubris and personality maybe interchangeable but one can have a relatively stable personality and high hubris or vice versa, each of which can start a war. Ambitions is distinctly cultural and historical, not necessarily a characteristic of a leader.
“Ambitions” of a nation is a very nebulous concept and ultimately driven by leadership. No one asked me about any of the military engagements that my country has participated in in my lifetime, so I would not say my ambitions are reflected.
This is one of those situations where I think I understand what you are saying, but ultimately the list is kind of rough. Especially because it overwhelmingly attributes war to personality quirks and some general sense of “mood.”
There's no point in fighting for territory for territory's sake, it's always about the resources, with a minor caveat for defensible geography.
Uh, US "first strikes" can't beat the Houthis and can't beat Iran. Track record: Remarkably poor. I'd go so far as to say that the entire US way of waging war -- its warfighting doctrine -- has been thoroughly discredited in light of this & the efficacy of top-dollar NATO weapons in Ukraine. (Good enough to hinder Russia, but far from good enough, and far from numerous enough, to win the war or even produce a decisive advantage.)
"You don't understand, we haven't gone Berserker Mode yet." (Sorry, couldn't resist.)
This is more of US being unreliable ally than them being incapable one.
[1] - https://en.wikipedia.org/wiki/Demographics_of_Ukraine#Popula...
Personally I see that as being terrifying.
Because you have a Western mindset. A lot of cultures can tolerate suffering much more and it is a mistake to think you can bomb people into submission when your own people can't handle gas prices going up.
Conventional war, though? Not in your wildest fantasies.
https://www.youtube.com/watch?v=SnEVnvWUJgM
> In the southern city of Kaohsiung, a group of retired Taiwanese naval officers invited the Americans to an ice-cream parlor and, over a round of sundaes, told them to stop preparing for war. They said that Taiwan possessed neither the determination nor the means to fight, and that the U.S. would never intervene to help. The defeatist attitude angered the two Americans, for what it said about Taiwan’s morale and for a more personal reason: Heginbotham’s son, who serves on a U.S. submarine in the Pacific, could be sent to fight in a war with China. “Sorry for my language, but what the fuck?” he recalled saying. “My son’s life is on the line to defend you guys.”
His complaint is that a hypothetical peaceful reunification would somehow be more dangerous to his active duty son than the war that he's advocating for.
I don't buy the world war theory. I do expect unrest regardless of AI outcomes, but that will be more internal than between countries.
But at this point that's just reiterating what has been reiterated many times with zero new insights.
I still think right now the cyborg approach is the best (like it used to be for a while in chess), even fable 5.2 needs steering, but the amount of steering I have to do now vs the amount of steering I had to do a year ago is significantly down. If the rate of improvement remains I can see fable 8 being able to be told "make this codebase more maintainable so we can fix the tech debt" with not a lot of supervision, and if we get there all bets are off on the future of coding as a career.
I wasn't really talking about tech at all. Maybe that was a bit confusing or misleading.
What you're saying is probably going to happen, but even with software having eaten the world, there is a lot more to the world than software.
So then the US restricts HW sales. Cool, the world makes it up in volume by running distributed inference or training just like Deepseek did. Components are expensive right now, but that, much like the tech debt, will ultimately be addressed.
There are apparently ~50 edge computing startups and ~100 AI HW startups right now. I would have said none of them stood a chance against Nvidia, but now that the federal government has commenced punching itself in the face over AI as they attempt to stop an avalanche by grabbing pebbles, anything goes 3-5 years from now when Nvidia's current roadmap runs out. And it's a lot worse than just the loss of the CEO, it's all the wasted energy by their staff jumping through hoops to prove to a bunch of soundbite chasing technologically ignorant Luddite politicians once again that Nvidia is not the Enron of technology. At worst, they were playing by the letter of the law as the violated the spirit, just like every single tax-dodging multimillionaire does annually.
None of this leads to WW3, but I think it leads to us falling behind for good. And given our choice of leadership and our geopolitical shenanigans, that's probably for the best.
AI is propping up what would otherwise be a harsh recession + we have (somewhat, in modern history) unprecedented (lack of) distribution of wealth.
That is the main story I am seeing, and in that story, software is at best a catalyst but likely pretty much irrelevant.
I personally don't want to live in a world where China is the global superpower. I also think nuclear war is a worse alternative (and I think the US agrees) - but I hope Western countries understand they need to build (energy, civil infrastructure, compute, manufacturing, etc.) if they don't want to pick between these two options.
China's primary food imports from the US are livestock feed (soy and corn) which Brazil could replace given some time. Don't forget that China is also investing significantly in Africa which has multiple countries working on soy production (no coincidence). The output isn't there yet, but could be close in 10-20 years.
In terms of long term strategy I find it hard to see any future where China hasn't already outplanned the US.
I see essentially no discussion of the other half of the equation, what we might call “the means of consumption”. Everyone is operating with a default unexamined assumption that consuming is intrinsically human, that “consumers” and “people” are synonyms. The closest anyone seems to get is the “who will buy your stuff if we don’t have jobs anymore to pay for it?” -> “universal basic income” -> “from all those famously generous tax-paying capitalists?” cul-de-sac.
Nobody’s talking about, what if AI can just build a better consumer, too? When humans are left without an ecological niche in the economy, what then? Like the economic equivalent of a flightless bird on a remote island with no natural predators encountering an invasive species for the first time, we would go the way of the dodo.
the best outcome in this case is general human incompetence
Which has repeatedly bore fruit... The "it's just marketing crowd" of the past year has been dead wrong. The Bun Rust rewrite, the cybersecurity hacks from Fable/Mythos+ class LLMs, people want to believe "it's just marketing" because it's so much scarier if the alternative is true. Anthropic and frontier lab employees are earnest and correct about the world we're going into...
"I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
Do not underestimate the power of this technology. These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger." -Jacob Coxon
Publicly, Russia has ~5k warheads. Public reports of US interceptors are on the scale of 10s. The only defense today against a decapitating first strike is MAD
Here's two:
https://www.afp.com/en/under-fire-over-ai-effective-altruism...
https://www.france24.com/en/live-news/20261002-effective-alt...
Holden Karnofsky, a prominent figure in EA, was a roommate of Amodei, and is now married with his sister.
And regarding AI, I think that the belief is that AGI or whatever is inevitable, and it's less bad if they do it first, because theirs will be less bad; and they see doing it as so important that everything can be justified.
The fanatisms of the 21st century are based on reasoning gone haywire (and maybe on psychotropic drugs).
As the typical fanatism, they convinced themselves to be so much in the right that anything is justified.
Crazy and scary.
So, with no reasoned argument, this guy somehow predicts the end of the world is coming and its effectively inevitable?
And people on this site take this couple of paragraphs at face value because... no real reason?
The Second World War didn't end the world, neither did the Cold War, and the world is now far more economically and politically intertwined than it ever was.
Do you think Russia really wants to nuke London? Do you know how many Russian oligarchs have apartments and houses in London where their extended families live? Similarly, do you know how many Chinese people live in London or how much Chinas economy is tied to Europe and the US?
If the world didn't end in the Cold War, it certainly isn't going to end now, and especially over AI, which although a cool tech, is incredibly far from this mythical singularity. Have you seen modern robots? Have you seen LLMs trying to do anything that isn't easily verifiable (with software)? E.g. try and create new drugs? Or design new materials, machinery, or complex systems like a fusion reactor.
Most people on this site are programmers and as such they are in a tiny bubble, obsessed with AI now its equal to them at the very act they have depended on for their entire careers.
But no, the end of the world is not coming any time soon. The rich Americans want to stay rich, the rich Russians want to stay rich, and the rich Chinese want to stay rich, and bombing each other and dying in fire isn't part of their agenda.
Not trying to be harsh, just a pattern I noticed.
Childless AI doomers may be gaining on them, but are relatively small.
What does worry me a bit is that the people running these AI companies _think_ they're summoning some earth-shattering angel of death, but they are so self-righteous (literally) that they think it's actually good that they're doing this.
Stupid software bugs were always the thing that's gonna bring doom. Just thought it was that guy that believes to be a genius and never makes mistakes instead of some computer hallucinating.
US and Russia to oppose talks on limiting autonomous weapons
https://www.reuters.com/world/us-russia-oppose-talks-limitin...
The answer: there still will be an economy, you just won't be a meaningful part of it.
Or Elon is right and somehow we'll all just "be rich". I'm not sure where that money's going to come from - he certainly isn't going to just start handing it out.
I mean, that's the ultimate power move. When nobody has anything, and you got some, and there's nobody to dictate you what to do with it, and you get to hand pick the winners and the losers.
Which has been a thing for a while. Asset wealth gain has outpaced income from labor, and the gap is now becoming more and more insurmountable, AI or not.
History is clear: Humans care about humans, and will on average act to correct social imbalances.
An economy with Elon and Amodei racing space rocket cars while large numbers of the population live off food stamps will be rejected.
Someone will just fly a drone into their houses.
Here I am, stuck in the middle with you
You see, this higher intelligence is so alien to us we can't comprehend how it works, but its timescale is completely different from ours. We think "weird default shortcuts due to an obscure keyboard layout to incapacitate people after a few generations" is a lot of time, but that's peanuts to AI.
The audio equipment manufacturer?
I have observed that the humble photon cannon rush is very effective. If not spotted it's usually a really bad time. Long term, is pretty obvious what happens here. If we get there.
Who dominates the gravity well that we all exist inside right now?
At some level it's never really been fair that I get paid so much more than them to do far less physically demanding work, and I suspect that shape of the economy might be coming to an end.
So “best case” (no destruction, annihilation etc. scenarios), global GDP goes down and we revert to a pre-≈1950s economy, in terms of job distribution across the farming/manufacturing/services sectors? Still doesn’t sound all that rosy to me, but I don’t know.
Edit: Who am I misunderstanding? A downvote with no explanation doesn’t help me get your point.
Economic efficiency is up for debate, but neither side is decisive militarily.
Worth pointing out that the Chinese Navy has also massively expanded over the last few decades and continues to expand - they aren't at parity with the US yet but they are able to contest the South China sea (which was the first goal) especially when backed with missiles launched from the mainland.
The trajectory is set there and China has vastly more yard capacity than the US (they are in the #1 spot, they out build the #2 and #3 spots who are US allies (Korea and Japan) so any (limited) shooting war where ships are been sunk comes down to who can sustain and replenish the losses the fastest, they are both nuclear powers so how wide spread the attacks would be on each others ship production is an open question - it's a risky game either way.
The US navy was built to it's current size during peace time when it's much simpler to build slowly and maintain a big fleet, what I don't think they can do is replace the losses they'd take in that war fast enough.
That in the end is the deterrent, you don't have to guarantee a win to deter someone you just have to make the price so high they don't want to enter the fight in the first place.
What would he even know about geopolitics or sociology to give him any credibility outside of his domain?
I.e. he is asserting an outcome "armageddon", but he provides absolutely zero argumentation, reasoned hypotheses, debate, etc, as to why it would happen.
"That which can be asserted with no evidence can be dismissed with no evidence."
Therefore, I assert that there is no pending end of the world scenario, and I don't need to provide any reasoning for that, since George Hotz doesn't either.
That said, I think the EA people need to touch grass too.
And while I don't think much of people myself, I don't think that immediately leads to WW3.
Humanity is something like a great inert blob. It takes tremendous effort and charisma and trickery and cajoling and organizing and "nudges" to get the blob to move. Fear is a great motivator, but if we do not fear, we cannot be made to move.
There is no such thing as a spontaneous uprising in humanity. The flash mob or corner protest was organized. Fads like the 6ft social distancing floor stickers were printed and placed everywhere practically overnight and done with terrific social engineering forethought.
When one side is on the verge of winning everything, perceived or real, the other side will just pull out higher risk/reward strategies. Yes I also play RTS.
I can imagine many doom scenarios but I just don't see a single positive. The AI's will do our jobs and our overlords will give us free food and housing? Keeping us around as pets? Out of sheer good will and because of the cuteness of our eyes?
I just think the world is too complicated, passionate, and resilient for any one dimensional fantasy about evil godlike computers to feel like anything more than self-satire.
The doom must be gently, respectfully shamed.