I'm trying to understand why people are spinning up so many k8s clusters that they need a tool to do it for them?
I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
krisknez 1 hours ago [-]
You're cool if you manage your own K8S cluster.
vasco 47 minutes ago [-]
It's applied big brain memetics. k8s turned pet servers into cattle. People then do the next step and want to treat their clusters as cattle as well. Also it has a bit of the "can it run DOOM" vibe to treat whole k8s clusters like this.
skullone 5 hours ago [-]
I went RKE2, k3s is nice, but a little too minimal for my tastes. With a few hundred MB ram used, I've got an internal container registry, openbao for secrets, caddy for edge TLS, rabbitmq, and powerdns for exposing k8s ingress. Plus all the standard network policies, which while verbose, gets me nearly all the way there of traditional firewalls and networking.
doctoboggan 5 hours ago [-]
I used this for a bit a few years ago but eventually needed something that was hard or impossible in k3sup and just went to using the k3s tools directly. My deployment script actually got simpler after removing k3sup.
Also, fun fact, k3sup is pronounced "ketchup" according to the README[0]
The pronunciation ketchup is somewhat unfortunate as a popular backup operator for k8s, k8up, also claims this.
debarshri 6 hours ago [-]
You can pretty install it without ssh under 60s. The fun starts after it has been installed.
We have been running into lot of issues at production with k3s. There I embarked on journey to writing a kubernetes compliant and equivalent platform in rust with the help of claude [1]. It is a fun little project for now, still figuring out stuff, idea is to keep it minimal and single binary every embedded including CNI, and support various runtimes like docker, containerd etc but also wasm, vms and also jvm.
We do, let me check with my team and post it here.
There were many issues. On top of my mind was, after a DR drill where in a VM was booted, node did not join the cluster. Apart from that bunch of issues due to etcd, longhorn.
Another major one was the CNI stopped work for a particular node. Garbage collection for images was another, we labelled the images, it would still remove then from the node.
Bunch of these kind of issues when our requirement is fairly straightforward. Therefore we are working towards a strip down version.
There is lot of operation complexity in general and most of us can do without.
Foobar8568 4 hours ago [-]
Meanwhile our architecture team that surely supported 0 real life k8s went with no vendor, on premises deployments, claiming it was as easy as booting a VM, after 2y, there is 2 apps running and supposedly all future apps will be deployed on that cluster.
I cannot wait for the end of this month to leave that place.
debarshri 3 hours ago [-]
Anything real world tech is hard.
We are hiring, btw.
ggiesen 5 hours ago [-]
I can bootstrap an entire RKE2 VM (VM + RKE2 + join cluster) in like 5 mins with Salt (although I have no reason to think you couldn't do it with Ansible).
It's a cool project, but I didn't think the K3s part was the hard part.
caymanjim 5 hours ago [-]
What's the point? You can bootstrap k3s with "curl -sfL https://get.k3s.io | sh -". If you need to do that over ssh it works just fine. If you're doing it on multiple hosts, you should be using Ansible.
maxdo 6 hours ago [-]
the best part of k8s is network, most of agentic systems presume no network , since it's a security concern, what are scenarios when you'd like to spin up k3sup?
tgrowazay 8 hours ago [-]
I use official ‘ansible-playbook k3s.orchestration.site -i inventory.yml’ and it installs k3s over SSH and adds it into my kubectl context, all under 60s too.
verdverm 6 hours ago [-]
I have just been `ssh ... -- k3s.sh ...`, been meaning to ansible my homelab
cieulyyy 3 hours ago [-]
[flagged]
Rendered at 09:27:17 GMT+0000 (Coordinated Universal Time) with Vercel.
I have one. And it's managed. I don't think there's significant cost savings to going unmanaged, but maybe. Even so, why would I need a ton of them?
Also, fun fact, k3sup is pronounced "ketchup" according to the README[0]
[0]: https://github.com/alexellis/k3sup/blob/master/README.md
We have been running into lot of issues at production with k3s. There I embarked on journey to writing a kubernetes compliant and equivalent platform in rust with the help of claude [1]. It is a fun little project for now, still figuring out stuff, idea is to keep it minimal and single binary every embedded including CNI, and support various runtimes like docker, containerd etc but also wasm, vms and also jvm.
[1] https://github.com/debarshibasak/superkube
There were many issues. On top of my mind was, after a DR drill where in a VM was booted, node did not join the cluster. Apart from that bunch of issues due to etcd, longhorn.
Another major one was the CNI stopped work for a particular node. Garbage collection for images was another, we labelled the images, it would still remove then from the node.
Bunch of these kind of issues when our requirement is fairly straightforward. Therefore we are working towards a strip down version.
There is lot of operation complexity in general and most of us can do without.
I cannot wait for the end of this month to leave that place.
We are hiring, btw.
It's a cool project, but I didn't think the K3s part was the hard part.