Hi @Srikanth,

Have you tried a restart of the MWS to see if it address the concern?

Cheers!

Novishan

Yes, a restart and reconfiguration of the application has been tried already. 

Hello @Srikanth,

This issue happens due to failed Stream messages. could you please check if there are any failed JMS Stream messages available in the WebLogic Admin Console? If there are any available please delete them first.

The logs are getting bigger and bigger due to the repetition of mentioned error and  it causes for a huge CPU consumption of both MainServer1 and MainServer2. As a solution, you can change the value of the parameter “Redelivery limit” by logging into the Middleware Server Admin Console. Please find below screenshot.

-Login to IFS Middleware Server Admin Console.

-Navigate to Common>JMS Modules> StreamJMSModule> StreamsJMSTopic.

-Change the value of "Redelivery Limit" from -1 to 5 -Restart the Middleware Server for the change to take effect.

The  default value of the field “Redelivery Limit” has been set into -1, which means unlimited retries. This would have resulted in getting logs written and consuming a large number of CPU usage as well as Logs growth. By changing the value, set to “Redelivery Limit”, the stream message would drop after several tries and the CPU utilization should come down. Hence, change the value into like 5.

Also please follow below steps as well. 

1). Stop MainServer1.

2). Delete  *.dat files under

wls_domain\<INSTANCE_NAME>\servers\MainServer1\data\store\default

wls_domain\<INSTANCE_NAME>\servers\MainServer1\data\store\diagnostics

3). Restart the server and check if the error is still available.

Please check the solutions 286167 and 281112 (UPD9) as well.

Cheers!

Hi @Srikanth,

As Imal nicely explained, 

This issue is caused by an incorrect stream message which doesn't specify a to user or specifies an incorrect user. This will cause the Stream app running on the server to throw an error which is then written to the log file. 

Cheers!

Novishan

Hello @Srikanth,

This issue happens due to failed Stream messages. could you please check if there are any failed JMS Stream messages available in the WebLogic Admin Console? If there are any available please delete them first.

The logs are getting bigger and bigger due to the repetition of mentioned error and  it causes for a huge CPU consumption of both MainServer1 and MainServer2. As a solution, you can change the value of the parameter “Redelivery limit” by logging into the Middleware Server Admin Console. Please find below screenshot.

-Login to IFS Middleware Server Admin Console.

-Navigate to Common>JMS Modules> StreamJMSModule> StreamsJMSTopic.

-Change the value of "Redelivery Limit" from -1 to 5 -Restart the Middleware Server for the change to take effect.

 

The  default value of the field “Redelivery Limit” has been set into -1, which means unlimited retries. This would have resulted in getting logs written and consuming a large number of CPU usage as well as Logs growth. By changing the value, set to “Redelivery Limit”, the stream message would drop after several tries and the CPU utilization should come down. Hence, change the value into like 5.

Also please follow below steps as well. 

1). Stop MainServer1.

2). Delete  *.dat files under

wls_domain\<INSTANCE_NAME>\servers\MainServer1\data\store\default

wls_domain\<INSTANCE_NAME>\servers\MainServer1\data\store\diagnostics

3). Restart the server and check if the error is still available.

Please check the solutions 286167 and 281112 (UPD9) as well.

Cheers!

I will give this a try now and keep you posted. Thank you!

@Imal Thiunuwan The recommended fix is working nicely and we have applied it to the Production instance as well. Thanks for that.

Before I mark your response as the Best Answer, I would like to know why the Stream Messages failed in the first place. I don’t think the client is using this functionality. 

Is this a known issue in UPD8 which has been fixed in UPD9?

Any additional information will be helpful.

Regards, Srikanth

Hello @Srikanth ,

Yes, these messages are not created or assigned by a user. These are generated from WebLogic end. There is a fix available in UPD9; the solution 281112 (Bug ID: 153502). Furthermore, to figure out where this message is originating from you can execute the following query in the environment and see if it has any results.

SELECT * FROM fnd_stream_tab t 
WHERE t.to_user IS NULL OR
NOT EXISTS (SELECT 1 FROM fnd_user_tab u where u.identity =  t.to_user)

Cheers!