Memory leak via node SSH shell
Eugene Aseev
Hi,
We have an issue with Corda 4.3 OS leaking memory on the SSH thread. We're running regular health checks by connecting to the node shell via SSH and running run exit command. After some time node memory consumption grows and node reboots (you can see these cycles on the chart attached). Also, a brief investigation shows that every time we connect to the node over SSH, a new file descriptor is opened. So the number of opened files grows all the time. The node is running in Docker under Kubernetes with the following JVM args: -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap. Any ideas on how to mitigate this? Regards, Eugene Aseev |
|
Przemyslaw Bak
Hi Eugene,
does the same happen when you do the health check without docker/k8s ?
Regards, Przemek
From:
<corda-dev@groups.io> on behalf of "ea via Groups.Io" <ea@...>
Hi, |
|
Eugene Aseev
Yep, just tried locally, same result. Descriptor number keeps growing when I connect and exit using regular ssh client.
Regards, Eugene Aseev
From: corda-dev@groups.io <corda-dev@groups.io>
On Behalf Of Przemyslaw Bak via Groups.Io
Sent: Tuesday, 7 January 2020 4:27 PM To: corda-dev@groups.io Subject: Re: [corda-dev] Memory leak via node SSH shell
Hi Eugene,
does the same happen when you do the health check without docker/k8s ?
Regards, Przemek
From:
<corda-dev@groups.io> on behalf of "ea via Groups.Io" <ea@...>
Hi, |
|
That'd be a bug. I've filed it here:
https://github.com/corda/corda/issues/5842 https://r3-cev.atlassian.net/browse/CORDA-3520 |
|