the features worth mentioning imho are the different storage backends and their features. Using btrfs, lvm or zfs there is some level of support of thin copy provisioning and snapshotting. I believe btrfs/zfs have parity in terms of supported operations. Cheap snapshots and provisioning of both containers and VMs using the same tool is pretty awesome.
I personally use lxd for running my homelab VMs and containers
gavinray 1 hours ago [-]
Can someone explain the usecase for this?
Is this for people who want to run their own cloud provider, or that need to manage the infrastructure of org-owned VM's?
When would you use this over k8s or serverless container runtimes like Fargate/Cloudrun?
Levitating 1 hours ago [-]
There's no particular usecase, though I do know of a company whose entire infrastructure is maintained within incus.
I personally use it mostly for deploying a bunch of system containers and some OCI containers.
But anyone who uses LXC, LXD, docker, libvirt, qemu etc. could potentially be interested in Incus.
Incus is just an LXD fork btw, developed by Stephane Graber.
clvx 46 minutes ago [-]
Who also developed LXD and contributed to LXC. I wouldn’t say it’s just a fork but a continuation of the project without Canonical.
rascul 6 hours ago [-]
A little bit of context about where Incus came from:
Proxmox feels like a more apt comparison, as they both act like a controlplane for KVM virtual-machines and LXC containers across one or multiple hosts.
If you are interested in running kubernetes on top of incus, that is your kubernetes cluster nodes will be made up of KVM or LXC instances - I highly recommend the cluster-api provider incus https://github.com/lxc/cluster-api-provider-incus
This provider is really well done and maintained, including ClusterClass support and array of pre-built machine images for both KVM and LXC. It also supports pivoting the mgmt cluster on to a workload cluster, enabling the mgmt cluster to upgrade itself which is really cool.
You may have a point there that k8s is not meant for single machines but that’s not a hard rule, more like a “why would you want to” you can absolutely run single node Kubernetes.
Also strictly speaking incus is not a container nor vm runtime, it’s an orchestrator of those things.
danofsteel32 4 hours ago [-]
Incus is great when developing ansible playbooks. The main benefit for me over docker/podman is systemd works out of the box in incus containers.
anonfordays 3 minutes ago [-]
What makes it better than Vagrant for this use-case?
mekster 3 hours ago [-]
Not to mention the easy to use web UI.
Levitating 1 hours ago [-]
To be fair until recently the web UI was just 15 patches placed over the LXD UI.
actinium226 3 hours ago [-]
I went through the online tutorial, but I'm not really seeing how it's different from docker?
skydhash 2 hours ago [-]
Instead of ephemeral containers, you have instances that are like VM (and incus can manage VM via qemu), so pretty much everything you would use a VM for, but if you do not need the kernel separation. It's more similar to FreeBSD jails than to docker.
Levitating 1 hours ago [-]
It's a difference between system containers and application containers.
LXC containers used in incus run their own init, they act more like a VM.
However incus can also execute actual VMs via libvirt and since recently even OCI containers like docker.
burnt-resistor 4 hours ago [-]
Nothing about resource (net, io, disk, cpu) isolation, limits, priorities, or guarantees. Not the same as a type 1 hypervisor. These qualities are needed to run things safely and predictably in the real world™, at scale. Also, accounting and multitenancy if it's going to be used as some sort VAR or VPS offering.
tok1 3 hours ago [-]
Fun fact, Incus is being used as underlying infrastructure for the NorthSec CTF, i.e. in an "as hostile as it can get" environment. If you have close to a hundred teams of hackers on your systems trying to break stuff, I think it speaks for Incus and its capabilities regarding isolation and limits.
In case you are interested, Zabbly has some interesting behind-the-scenes on Youtube (not affiliated).
The only tool I found which allows to easily spin up pre-configured VMs without any gui hassle
63stack 6 hours ago [-]
How do you handle updating the machine that Incus itself runs on? I imagine you have to be super careful not to introduce any breakage, because then all the VMs/containers go down.
What about kernel updates that require reboots? I have heard of ksplice/kexec, but I have never seen them used anywhere.
dsr_ 5 hours ago [-]
As with any such system, you need a spare box. Upgrade the spare, move the clients to it, upgrade the original.
loloquwowndueo 4 hours ago [-]
But then the clients have downtime while they’re being moved.
pezezin 3 hours ago [-]
I don't know about Incus, but on ProxMox the downtime when moving a VM is around 200 ms.
pylotlight 4 hours ago [-]
Isn't that the exact problem that k8s workloads solve by scaling onto new nodes first etc?
No downtime required.
loloquwowndueo 4 hours ago [-]
Right but incus is not k8s. You can stand up spares and switch traffic, but it’s not built in functionality and requires extra orchestration.
pm2222 1 hours ago [-]
Should lxc user migrate to incus?
goku12 1 hours ago [-]
Short answer: No. Long answer: Depends upon what you use lxc for.
Incus is not a replacement for lxc. It's an alternative for LXD (LXD project is still active). Both Incus and LXD are built upon liblxc (the library version of lxc) and provide a higher level user interface than lxc (eg: projects, cloud-init support, etc). However, lxc gives you fine grained control over container options (this is sort of like flatpak and bubblewrap).
So, if you don't need the fine grained control of lxc, Incus may be a more ergonomic solution.
PS: Confusingly enough, LXD's CLI is also named lxc.
Levitating 1 hours ago [-]
an lxd user should
manosyja 9 hours ago [-]
What can this work with? It says „Containers and VMs“ -
I guess that’s LXCs and QEMU VMs?
nrabulinski 8 hours ago [-]
Yes, it uses QEMU under the hood for VMs and runs LXC containers. But also, since recently, you can run docker images in it. Very handy, especially since it has 1st class remote support, meaning you can install only the incus client and when doing `incus launch` or whatever, it will transparently start the container/vm on your remote host
Lightkey 6 hours ago [-]
Not to be confused with the cirrus7 incus[0], which are fanless PC models based on the ASRock DeskMini series that I'm using right now.
Is there some kind of Terraform/Pulumi integration to make it easy to deploy stuff to some VM running Incus for my deployments? Or I'm missing the point of what Incus is for?
Levitating 1 hours ago [-]
You could use cloud-init
Rendered at 16:38:34 GMT+0000 (Coordinated Universal Time) with Vercel.
I personally use lxd for running my homelab VMs and containers
Is this for people who want to run their own cloud provider, or that need to manage the infrastructure of org-owned VM's?
When would you use this over k8s or serverless container runtimes like Fargate/Cloudrun?
I personally use it mostly for deploying a bunch of system containers and some OCI containers.
But anyone who uses LXC, LXD, docker, libvirt, qemu etc. could potentially be interested in Incus.
Incus is just an LXD fork btw, developed by Stephane Graber.
https://lwn.net/Articles/940684/
[0]: https://tadeubento.com/2024/replace-proxmox-with-incus-lxd/
If you are interested in running kubernetes on top of incus, that is your kubernetes cluster nodes will be made up of KVM or LXC instances - I highly recommend the cluster-api provider incus https://github.com/lxc/cluster-api-provider-incus
This provider is really well done and maintained, including ClusterClass support and array of pre-built machine images for both KVM and LXC. It also supports pivoting the mgmt cluster on to a workload cluster, enabling the mgmt cluster to upgrade itself which is really cool.
I was surprised to come across this provider by chance as for some reason it's not listed on the CAPI documentation provider list https://cluster-api.sigs.k8s.io/reference/providers
https://linuxcontainers.org/incus/docs/main/explanation/clus...
You may have a point there that k8s is not meant for single machines but that’s not a hard rule, more like a “why would you want to” you can absolutely run single node Kubernetes.
Also strictly speaking incus is not a container nor vm runtime, it’s an orchestrator of those things.
LXC containers used in incus run their own init, they act more like a VM.
However incus can also execute actual VMs via libvirt and since recently even OCI containers like docker.
In case you are interested, Zabbly has some interesting behind-the-scenes on Youtube (not affiliated).
What about kernel updates that require reboots? I have heard of ksplice/kexec, but I have never seen them used anywhere.
Incus is not a replacement for lxc. It's an alternative for LXD (LXD project is still active). Both Incus and LXD are built upon liblxc (the library version of lxc) and provide a higher level user interface than lxc (eg: projects, cloud-init support, etc). However, lxc gives you fine grained control over container options (this is sort of like flatpak and bubblewrap).
So, if you don't need the fine grained control of lxc, Incus may be a more ergonomic solution.
PS: Confusingly enough, LXD's CLI is also named lxc.
[0] https://www.cirrus7.com/produkte/cirrus7-incus/