Lots of details that teams miss when tuning the JVM on Kubernetes:
1. Default heap size varies: it can be 50% of the available memory for any amount of available memory up to 256mb; ~128mb for anywhere between 256 and 512; and then 25% for anything above 512mb;
2. The JVM will still do parallel threads even when CPU limit is 1,000 or below , which triggers CPU throttling much faster. Threads such as GC (even when it is SerialGC), ForkJoin pools, Database connection pools, HTTP server, and so on.
3. The JVM behaves MUCH better with 2,000 or more milicores, and a well sized heap. Having very small containers and dozens of replicas is worse than having slightly bigger containers (cpu/memory limits) and fewer replicas, even with a different ratio (e.g. 6 replicas will perform worse than 3 replicas, even when aggregated CPU and memory limits are the same).
I remember switching on G1GC for Elasticsearch back in 2014 and it was a total game changer for cluster stability (ES had plenty of other warts back then to counter the improvement). Cool to see it land as the default everywhere over a decade later.
cogman10 2 hours ago [-]
It's definitely gotten a LOT better since it was introduced. Java 17, in particular, really made G1 great.
But if you've not already looked at it for your usages, I suggest people check out both the parallel collector and ZGC. ZGC in particular has been getting a lot of very nice features. Semi recently getting fast heap freeing (to the OS). The parallel collector for small heaps can generally beat G1 pause times.
Where G1 really shines over the parallel collector is the fact that it regularly collects old gen. If your app has a lot of caches, then G1 is very good at keeping the VM from ending up with very long pauses.
pjmlp 44 minutes ago [-]
It is not everywhere, because there are other JVM implementations, with their own GC algorithms.
pancakeguy 42 minutes ago [-]
garbage collection will almost certainly be a relic of the past surprisingly soon. Fundamentally it exchanges efficiency for readability & developer experience, to things that do not matter when using AI to write code.
sealWithIt 15 minutes ago [-]
On the one hand, I agree with you; on the other, I also find reasonable that lower cognitive load may keep justifying the existence of GC-based languages even for AI.
xxpor 16 minutes ago [-]
Disagree. Maybe the naive business app use case will, but there are plenty of places where you want to implement effective GC regardless. Anywhere you batch frees to avoid calling it in a hot path is a GC. So it's not _always_ an efficiency sacrifice. Or, a different perspective is you always have to ask "efficient with what resource?" CPU time efficiency vs memory efficiency for example (an interesting case where the _money efficiency_ has obviously moved severely in the past year!).
ogogmad 35 minutes ago [-]
If true, it's sad. So much work and so much accumulated wisdom. It's like seeing a cathedral build by thousands of hands over dozens of years collapse in an earthquake.
jitix 14 minutes ago [-]
Disagree, we need to move ahead with progress.
Hobby and domain specific languages may still use GC but getting rid of the concept of GC is the right direction, along with removing different ways of doing things in a programming language. Go was right when it forced developers to use only pattern for loops, one pattern for error handling, etc.
IncreasePosts 22 minutes ago [-]
Doesn't this assume infinite abundance of compute? For most problems, using a GC is probably easier for AI than to track everything manually(just like with humans). Similarly, we could probably ask AI to output everything in optimized assembler code, but it will almost certainly burn more tokens getting to a solution than if we allow it to write in python or some other HLL
ahmedfromtunis 3 hours ago [-]
I love that they also have learn.java and inside.java.
They should make get.java point to the download page. Oh and docs.java would be very cool!
cyberax 1 hours ago [-]
And still no automatic heap sizing... Maybe next year?
karussell 59 minutes ago [-]
What do you mean? G1 definitely has automatic heap sizing, i.e. Xmx just defines the maximum it is allowed to use.
One great feature in Go (or Python, or Node JS) is that you don't have to care about setting the -Xmx. The runtime will just use more RAM as needed, and you can use normal mechanisms like cgroups to limit the maximum size.
Sid_Lmao 4 days ago [-]
Easy-to-miss detail: before JDK 27, the JVM only picked G1 with 2+ CPUs and ~1.8 GB of memory; otherwise, it silently used SerialGC. So a lot of 1- CPU Kubernetes pods have been on serial all along. Worth checking with -XX:+PrintCommandLineFlags before upgrading, since GC behaviour will change.
brunoborges 44 minutes ago [-]
This is correct. I have been advocating about this for years now [1].
I have also seen customers wasting resources across Kubernetes because their developers would eventually tune JVMs with lowercase `-xmx` flag, which the JVM ignores, and thus by default, the JVM only sets heap size to 25% by default (for => 512mb of available memory).
> So a lot of 1- CPU Kubernetes pods have been on serial all along
I don't think this is true.
Setting your CPU limit to 1 in Kubernetes does not mean the container sees a single logical processor. It only means the kernel allows it to use 1 core worth of CPU _time_.
If your k8s node is an 8 core node, Java will still see 8 logical processors. It just means you can only use 1/8th of each, or all of one, or some combination in-between.
I've never seen a k8s node running on a single core VM, as most of the cloud providers only use these for their free tier or shared core VM's for the lightest of light workloads.
rileymichael 28 minutes ago [-]
[dead]
citrin_ru 4 hours ago [-]
Isn't the reason for this that on a single core SerialGC is better (lower CPU overhead at least)?
brunoborges 43 minutes ago [-]
A 1,000 millicore CPU limit on Kubernetes does not mean the JVM can only use a single core. But that is indeed how the JVM interprets the limit: as a single core being available.
antonhag 3 hours ago [-]
Lower overhead yes - but G1 should have shorter pauses. And shorter pauses is probably worth it unless you run only background jobs in your jvms.
cogman10 2 hours ago [-]
Lower CPU and Memory overhead.
Serial will be your best choice if your app is constantly using 100% of that core or if you need to minimize the amount of memory overhead.
G1 is a better choice if you can be flexible with your memory and if you have a lot of periods in your app where it's below 100% CPU usage (maybe a lot of IO).
ch4s3 3 hours ago [-]
I'd imagine so. The kind of bookkeeping they're talking about here would negatively effect performance on a single core.
pjmlp 43 minutes ago [-]
Open JDK, not The JVM.
There are other implementations.
4 days ago [-]
_verandaguy 4 hours ago [-]
Tangential, but I really dislike that the `.java` gTLD was allowed to come about given it's also a reasonably popular file extension.
The `.zip` gTLD set a really bad precedent.
lmz 4 hours ago [-]
.zip? Not .com?
dbt00 2 hours ago [-]
The potential for confusion in 1985 wasn't really on anyone's radar.
dboreham 4 hours ago [-]
Old school.
Rendered at 19:33:05 GMT+0000 (Coordinated Universal Time) with Vercel.
1. Default heap size varies: it can be 50% of the available memory for any amount of available memory up to 256mb; ~128mb for anywhere between 256 and 512; and then 25% for anything above 512mb;
2. The JVM will still do parallel threads even when CPU limit is 1,000 or below , which triggers CPU throttling much faster. Threads such as GC (even when it is SerialGC), ForkJoin pools, Database connection pools, HTTP server, and so on.
3. The JVM behaves MUCH better with 2,000 or more milicores, and a well sized heap. Having very small containers and dozens of replicas is worse than having slightly bigger containers (cpu/memory limits) and fewer replicas, even with a different ratio (e.g. 6 replicas will perform worse than 3 replicas, even when aggregated CPU and memory limits are the same).
I've been advocating on this for years [1].
[1] https://www.youtube.com/watch?v=wApqCjHWF8Q
But if you've not already looked at it for your usages, I suggest people check out both the parallel collector and ZGC. ZGC in particular has been getting a lot of very nice features. Semi recently getting fast heap freeing (to the OS). The parallel collector for small heaps can generally beat G1 pause times.
Where G1 really shines over the parallel collector is the fact that it regularly collects old gen. If your app has a lot of caches, then G1 is very good at keeping the VM from ending up with very long pauses.
Hobby and domain specific languages may still use GC but getting rid of the concept of GC is the right direction, along with removing different ways of doing things in a programming language. Go was right when it forced developers to use only pattern for loops, one pattern for error handling, etc.
They should make get.java point to the download page. Oh and docs.java would be very cool!
edit: looks like you probably mean sth like this: https://openjdk.org/jeps/8359211
One great feature in Go (or Python, or Node JS) is that you don't have to care about setting the -Xmx. The runtime will just use more RAM as needed, and you can use normal mechanisms like cgroups to limit the maximum size.
I have also seen customers wasting resources across Kubernetes because their developers would eventually tune JVMs with lowercase `-xmx` flag, which the JVM ignores, and thus by default, the JVM only sets heap size to 25% by default (for => 512mb of available memory).
[1] https://www.youtube.com/watch?v=wApqCjHWF8Q
I don't think this is true.
Setting your CPU limit to 1 in Kubernetes does not mean the container sees a single logical processor. It only means the kernel allows it to use 1 core worth of CPU _time_.
If your k8s node is an 8 core node, Java will still see 8 logical processors. It just means you can only use 1/8th of each, or all of one, or some combination in-between.
I've never seen a k8s node running on a single core VM, as most of the cloud providers only use these for their free tier or shared core VM's for the lightest of light workloads.
Serial will be your best choice if your app is constantly using 100% of that core or if you need to minimize the amount of memory overhead.
G1 is a better choice if you can be flexible with your memory and if you have a lot of periods in your app where it's below 100% CPU usage (maybe a lot of IO).
There are other implementations.
The `.zip` gTLD set a really bad precedent.