Things are the way they are not because they would be difficult but because various systemic incentives and goals fought out each other, leading to the outcomes we then saw.
This balance these various forces fought out relied on a bunch of undocumented assumptions about reality. Every time one of those breaks, you get disruptions. LLMs broke quite a few of them, leading to a lot of CVEs lately.
dgoodell 20 minutes ago [-]
I think it was a business decision to make it too complex for humans. I don’t think fundamentals were prioritized as much as features, resulting in a fragile house of cards.
There’s a reason it doesn’t look like most engineering. If it was a bridge it would have collapsed long ago and killed people.
amelius 9 minutes ago [-]
The always changing requirements are what make it too complex for humans.
ssrc 4 minutes ago [-]
Maybe for some software systems it was. For most systems, the problem was just professional immaturity caused by the eternal September state the profession is in since the beginning.
ex-aws-dude 6 minutes ago [-]
I mean if you zoom out the whole lesson of AI/ML is “we are not smart enough to design this function”
yepyoukno 3 hours ago [-]
Many handle it just fine.
AIShill101 3 hours ago [-]
Yea I agree with the article wholly. This new era of AI assisted or driven coding is going to help us extend our capability way beyond anything humans could've built.
onion2k 7 minutes ago [-]
This new era of AI assisted or driven coding is going to help us extend our capability way beyond anything humans could've built.
For individuals, sure. I'm fairly sure we're all building things that we couldn't build ourselves if only because we don't have the time needed. For large scale projects like building an OS from scratch or a database engine that's a lot less likely to be true, simply because there are dozens of examples of humans doing that level of complicated work without AI already (the OSs and DB engines we have are very literal evidence of it.) They just need to work collectively. Every part of those projects is understood people a person even if no person understands the whole thing.
So the problem with doing the same sort of thing with AI becomes a question of whether someone can build the equivalent sort of project with AI by describing something more complex than they understand themselves. And even if they just ignore the complex details and let AI do that bit, how can they validate that the AI has actually done what they asked if they can't understand it? It might have taken a shortcut that works until it doesn't. I hope no one builds an aviation or a life support system by doing that.
Ultimately, I suspect we want to be able to understand the system even if it limits what we build. A system that fails in an edge case carries significant risk which people probably don't want to be liable for.
dingaling911 19 minutes ago [-]
At the moment, oversight is still heavily required.
Before, we used to write code and machines converted it to binary.
Now we will write designs and machines will convert it to code (and then to binary.)
There is a possibility of models taking over design for somewhat run-of-the-mill applications with minor architectural tweaks. This is the area that "vibe coding" is currently dominating. How many vibe coded apps, written by others, are you using? None. So that tells you something.
At best, models are currently at VB6 level: domain experts can do a lot of good. But they could've done it with VB6 as well. And MS Access. And Excel.
I am not sure how we would give the models an idea of taste and beauty because that is often what differentiates good/bad from great designs.
Rendered at 21:00:34 GMT+0000 (Coordinated Universal Time) with Vercel.
Things are the way they are not because they would be difficult but because various systemic incentives and goals fought out each other, leading to the outcomes we then saw.
This balance these various forces fought out relied on a bunch of undocumented assumptions about reality. Every time one of those breaks, you get disruptions. LLMs broke quite a few of them, leading to a lot of CVEs lately.
There’s a reason it doesn’t look like most engineering. If it was a bridge it would have collapsed long ago and killed people.
For individuals, sure. I'm fairly sure we're all building things that we couldn't build ourselves if only because we don't have the time needed. For large scale projects like building an OS from scratch or a database engine that's a lot less likely to be true, simply because there are dozens of examples of humans doing that level of complicated work without AI already (the OSs and DB engines we have are very literal evidence of it.) They just need to work collectively. Every part of those projects is understood people a person even if no person understands the whole thing.
So the problem with doing the same sort of thing with AI becomes a question of whether someone can build the equivalent sort of project with AI by describing something more complex than they understand themselves. And even if they just ignore the complex details and let AI do that bit, how can they validate that the AI has actually done what they asked if they can't understand it? It might have taken a shortcut that works until it doesn't. I hope no one builds an aviation or a life support system by doing that.
Ultimately, I suspect we want to be able to understand the system even if it limits what we build. A system that fails in an edge case carries significant risk which people probably don't want to be liable for.
Before, we used to write code and machines converted it to binary.
Now we will write designs and machines will convert it to code (and then to binary.)
There is a possibility of models taking over design for somewhat run-of-the-mill applications with minor architectural tweaks. This is the area that "vibe coding" is currently dominating. How many vibe coded apps, written by others, are you using? None. So that tells you something.
At best, models are currently at VB6 level: domain experts can do a lot of good. But they could've done it with VB6 as well. And MS Access. And Excel.
I am not sure how we would give the models an idea of taste and beauty because that is often what differentiates good/bad from great designs.