Datastaging and multi-threading on CREAMCE¶

A bug has been reported related to the use of threads when downloading output files of CREAM-CE jobs with the JSAGA ((JobImpl)job).postStagingAndCleanup() method. In this case, when files have been downloaded, the thread that did the datastaging stays up for a while. If your program starts a high number of threads, this can lead to a OutOfMemory exception.

It is possible that this issue comes from the limits set on your system. Increase these values in the file /etc/security/limits.conf

• set or increase nofiles par user to 10240
• set or increase nproc par user to 4096

And call explicitely the garbage collector from your code:

system.gc();