And its this obsession with shipping things as fast as possible quality be dammed that got us basic weather apps that eat a gigabyte+ of RAM.

Yes, I remember those too. The costs of manual memory management were real and were not low.

But costs on the cloud are real too, especially now. I’ve been living in JVM land for a very long time, but now it’s especially clear how important lean services are. Especially now that the bar for writing lean code is so much lower: let the borrow checker figure it out, etc.

I just spent a couple days wringing out more performance/memory efficiency for our services. Nice gains to be sure, but it’s still so immensely wasteful compared to something well written running native. If it was my money, I’d be going native for sure.

(Cat reading newspaper) I should build a database out of pointers

Different people are different.

Some of us find production and optimization more interesting than marketing and distribution.

incredible way to put it!!

[dead]

An AI would have known that saying, “Hi, mom” in a professional post was a bad idea.

Build a system as simple as possible but no simpler.

An 1800s telegraph system doesnt work in the modem world, there is far too much communication and the system would just collapse into molten slag.

All those things you've listed are because we live in an adversarial world and I'd steal all your money off the telegraph wire if you tried it.

Having worked in hardware for a moment, everything we do in software is like this. Even C.

There are a whole series of blog posts from the Fishworks guys explaining why it could possibly be so hard to turn on one LED.

But Oracle probably deleted then so you'll have to find them on archive.org.

The steelman argumentation is probably that spaghetti doesn't matter to LLMs and no humans will read that code anyway.

Do you feel like this is a very meaningful comment? Will someone go "mm yes, actually it's spaghetti, I didn't think of that.."

Are you trying to succinctly say AL'S spaghetti code outweighs the benefits of what it produces quickly?

If you're not saying it outweighs it, what are you saying?

I mean you can just pause and refactor regularly. I tend to use every ~4th session as an opportunity to refactor, adjust interfaces, break overly large modules into smaller ones and suchlike.

The point isn't that "doing math" is safe. Auto-research solves one target variable in one system, doing it at scale where say one developer is SME for the agentically manged 200 microservices down the line, heh I mean you certainly can, but good luck with that token cost of auto-research when that problem space is O(microservice^2). I point at that example yesterday of that optimized database memory with the comments pointing out that the specific problem fit in memory, over optimized and didn't generalize. The problem isn't the work, but the rework. A historian should know that new solutions to problems doesn't lead to "no problems ever again" but only problems with barriers that the new solution doesn't solve.

Those machines with GPUs still need RAM of their own, and they generally want large caches to avoid SSD penalties. You even see this spill out in the form of costs for KV cache in <1min, 5m, 1hr rates etc.

The majority of inference actually does happen in cpu.

Well I looked it up; nginx, apparently, uses ketama -- it's a ring-style hash probably works better for web backends than the jch above, as when given [0,1,2,3] and replacing the server in slot 1 you're going to have a lot of hash moves. With ketama, you'd only have the '1' hashes moving. You can't really beat google's for brevity, though.

I have used Google's jump hash. As I recall, one of the main differences is that jump hash doesn't have a mechanism to remove targets, eg, a server dies and you don't want to route requests to it. Traditional consistent hashing can do that. I guess if you had 4 servers, server #4 dies, then you can go back to 3 servers by just changing num_buckets from 4 to 3. But if server 1 dies, you can't.

Jump hash does allow adding more targets and preserves the property that most request targets stay the same when adding a new target, so if you had 3 targets and add a fourth, ~8% of the requests that would have been sent to targets 1-3 are sent to target 4, evenly chosen from servers 1-3.

Nah, AI code is black or white, and whether the code is good depends on your religion.

AIded development?