Solved

OutOfMemoryError: Java heap space When processing application message FTP reader

  • 20 October 2023
  • 2 replies
  • 137 views
Takesha Kaluarachchi
Rank
Userlevel 2
Badge +5

Problem:

FTP reader cannot proceed with OPTIMITY INTEGRATION - java.lang.OutOfMemoryError: Java heap space error in application message.

checked int server logs ifsconnect,intserver and other logs and could find "Caused by: java.lang.OutOfMemoryError: Java heap space" error on intserver logs,

  • Checked the heap dump and could not find heap issue.

  • in this generated xml file
    one xml contains 75mb and other one is 56 mb,
    however file reader can read, For a 2GB heap space can process about 100 MB file. For larger files will have to increase the heap space allocated for the Integration Server accordingly.

  • This issue was resolved by increasing int server memory up to 8 gb (both int servers 16gb) and that was processing file with 8GB heap memory. As of now this issue is sorted temporally . But in future if this file grows beyond 56MB this issue can reoccur in same environments and tweaking the memory in-between managed servers no longer will be a solution

Suggestion to avoid this issue

  1.  file size can be reduced my compressing before writing the file to the FTP location. This has to be done by teh customer end
  2. optimize this optimity integration when writing file in to FTP location
  3. find an alternative way to archive this process without using ifs connect


Requested From RnD

Are there any limitations to IFS Connect, and are there opportunities for optimizing its performance and capabilities?"

icon

Best answer by udlelk 21 October 2023, 12:16

View original

2 replies

Takesha Kaluarachchi
Rank
Userlevel 2
Badge +5

Hi @udlelk ,

Could you please assist on this question?

U
Rank
Userlevel 5
Badge +14

@Takesha Kaluarachchi Please find the answers for your questions below:

IFS Connect is a light weight integration broker and it has known limitations with File Integrations. One of those limitations is, IFS Connect memory consumption is high when processing files. To read/process a file with the size 100 MB IFS Connect consume around 2 GB of a heap memory.

This sizing is possible assuming the messages are processing InOrder (one at a time). If the execution is Parallel which means 100 MB * (nX parallal threads), then the heap consumption will also increases along with the running parallels threads.  Also the other integrations which are running in the environment also consume the same heap memory which should be a considerable factor for the memory allocation.

With that IFS Connect Framework limitation in mind, R&D can recommend some changes/modification for the integration solution:

  1. Send files in smaller chunks – Require changes in file pushing solution also may require changes in data processing solution.
  2. Send files in compressed format – Require changes in both file pushing solution as well as in data processing solution

Please note that, optimizing IFS Connect Framework to utilize low heap memory in file integration requires a redesign of the framework which has to be done in a future IFS release.

Reply


Terms & ConditionsCookie settings