Skip to main content

Hi Everyone

Do you notice lastly problems with Odata pod? Time to time it restarted automaticaly after reaching some memory pick. This is quite strange because many of environments are affected by this(PROD, CFG and even DEV envrionments with poor usage). It is observed by many toast message which are shown in IFS Cloud.

 

Few hours ago I faced problem like this. It disapeared automatically after Odata pod is up again. By the way when I list the pods last restart was 24d ago.

I know that process is killed because one time after deep investigation I got this:

Sep  4 07:28:34 clouddemo kernel: :2415668.649318] Memory cgroup out of memory: Killed process 2070719 (java) total-vm:3125508kB, anon-rss:1542888kB, file-rss:38396kB, shmem-rss:0kB, UID:1000 pgtables:3400kB oom_score_adj:953

Hmm - 

It is a bit strange as it seems Linux kills the container as it exeeds it cgroup memory allocation.
The java heap (xmx) will never exceed the cgroup memory allocation. Possible java allocates too much metaspace memory… which has not upper limit.

is this an old version 23r2? … on a new Ubuntu 22.04? or is cgroup v2 enabled in Ubuntu?
Older java had a bug on cgroup v2, where it though the container had access to the entrie VM memory. Then Linux killed java once it used more than the cgroup allocation for that container.

Hi

Problem observed on 23.2.5 and 24.1.3. I was also redirected to problem mentioned here:

On one environment I changed suggested Excel parameter to check possible improvement.

I will back with more detail.

Reply


Terms & ConditionsCookie settings