Question

A lot amount of Out messages accumulated in IEE and not sent to mWO


Userlevel 3
Badge +9

Hi, All!
Our customer has issue on windows mobile client, when user made all the necessary actions with the work task and closed it, but there was no any progress appeared in IEE.
None of the failed transaction can be found for this user in IEE.
Also, Sync monitor page on mobile client shows "Idle” and the transactions list doesn't contain any. 
After the investigation we see a lot of Out messages (~11k for about a week) accumulated in IEE and not sent to mobile client back.
We, together with customer, restarted the tablet, closed and started the mobile client, but without any success.
Is there any suggestion what to do to move forward such amount of accumulated and not sent Out messages? Is it normal situation? How can we force resending these messages?
Thanks in advance!

BR,

DANMI


11 replies

Userlevel 6
Badge +14

Hi

If you see lot of pending out messages for a User/Device, then it means device has not managed to connect to the IEE for some reason or it managed by failed. Can you check the last access date in the Installed Device App screen? Also, you can perform a test with a different user to see data sync work as normal in the IEE. 

Are there any pending transactions in the device? Sometime if device has faulty transaction which can prevent data sync fully. In such case we need to identify the issue by looking at the server logs.

Regards

Kapila

 

Userlevel 3
Badge +9

Thanks a lot for the answer!

"...then it means device has not managed to connect to the IEE for some reason or it managed by failed.” - As we see from situation - the thing is that server cannot send any data to mobile client. User can work normally and proceed with existing tasks - accepting them, starting and closing. Also, the last access date is normal - always near to actual. So, it means that device has connection with IEE. Somehow there is no reverse connection - from server to device.

"Are there any pending transactions in the device?” - No, there are not. All is clear from this side. 

At some point we see the following error in mobile client log: "Server returned [401] Unauthorized”. And this error is followed by such warnings: "Broken is True- InvalidCredentials” and "Broken is True- RequiresActivation”. But at the same time it was possible for user to continue the work without initialization or re-initialization.

The main question here is - how to avoid such situation not allowing server to accumulate huge amount of out messages?

And the additional question - what to do if we've already got such situation? Is it possible to restore server/device connection and resend all the accumulated out messages?

BR,

DANMI

Userlevel 6
Badge +14

Ok, if you can do normal work in the MWO app and no pending transactions in the device means device connect to the server and executes them without any errors. The messages in server out queue could be from the different device same user used before but no longer active? This can happened if you have multiple devices. But latest App10 UPDs have a new functionality where server automatically move such installed device to a INACTIVE state and clear the out messages. Default is 7 days inactive. 

How many installed devices your user have? 

Userlevel 3
Badge +9

And again - thanks for participating in conversation!

As for out messages - all of them are from same device that is active at the moment. User has only one device at the same time. 

The main problem here is that part of user’s data not yet received by the server. It is gone from mobile client, but not received in IEE. We cannot find there is this information now.

All is good on mobile, but not so in IEE.

Out messages contain such significant information as server/client key exchange and we propose that server cannot write all received data due to key exchange is not over and data is not "trusted”.

Of course, this is only our guesses.

All advices/comments/proposals are appreciated!

BR,

DANMI

Userlevel 6
Badge +14

This is what I assume. Client sends transactions back to IEE and after that it try to download messages from server out queue. This includes any pending out messages or any receipts from the transactions previously sent. We call it Sing And Fetch messages which is the key for the data synchronization. if it stop working for some reason then we can see the issue where server out queue is growing. So I think this device might have a corrupted transaction which might holding the message signing process. 

Can we have a close look at IEE side? is this only for single user? 

 

Thanks

Kapila

Userlevel 3
Badge +9

What’re these Sign and Fetch messages?

How the device can have the corrupted transactions if we don't see any?

We have such situation for two more users, but their last access dates are from the middle of October.

Sure, we can have a close look at IEE side, but now this problem user has re-initialized his mobile client and all accumulated messages were gone.

Kapila, what is your advice if the similar case will appear?

What can we do from our support side to resend such out messages?

Or it is not possible to affect the situation?

Thanks in advance!

BR,

DANMI

Userlevel 6
Badge +14

Hi

“Sign And Fetch” functionality  means send pending transaction from the device to IEE and read any pending Out messages from IEE. 

Corrupted transaction means, device has a transaction which cannot be executed in the IEE because of a low level error. Such errors are triggered before they hit the IEE Server and Filed Transactions layer. So Not FT get created, but error can be return back to the device. No specific reason Most cases dues to bugs. So if you have lot of out messages in the server and lot of pending transactions in the device, then that could be the most obvious reason to stop sync data. 

First need to find the ROOT cause for the issue, then we can give you a workaround. Can you please send device Logs + J2EE logs (server). 

Admin has access to the server logs. Documentation: https://wit.ifsworld.com/f1docs/apps10/Foundation1/040_administration/400_core_server/010_middle_tier/010_middleware_server/020_troubleshooting/170_ifs_application_logging/default.htm

Device Logs

Mobile user have the option to send device logs via an Email.

 

Hope this helps for the next step.

 

Regards

Kapila

Userlevel 3
Badge +9

Hi, Kapila!

Here are device logs and J2EE logs (see in attachment).

BR,

DANMI

Userlevel 6
Badge +14

Thanks for uploading logs,

I can see the error ‘HttpCode: Unauthorized’ in client logs which is the last printed trace on 07th Nov. Can you please confirm you logged off and login again to the device app using PIN or password?

Also can you please confirm the first pending transaction in your device sync Monitor. Please embed a screenshot. 

Regards

Kapila

 

Userlevel 3
Badge +9

Hi, Kapila!

All customer users log in using the PIN. Authentication method is ADFS.

"Also can you please confirm the first pending transaction in your device sync Monitor. Please embed a screenshot.” What do you mean? If you are about the problem with a lot Out messages, then this case occurred on one customer user device. And this user has already done initialization - so, he doesn’t have any Out messages anymore. 

As for pending transaction - we didn't find any on the device sync Monitor while the problem persisted.

BR,

DANMI

Userlevel 6
Badge +14

Hi,

So this is not what we normally expect when we have stuck messages in the queue and the device. Normally we see messages in sync monitor which are pending and also IEE out queue same time. So this must be something else. If you do not see any waiting messages in device means device sends them to IEE without any issue. 

I think you will need to come through IFS support to het further help as information is not enough. Can you please upload pending messages image in IEE and also showing your Installed Device App screen in IEE. ?

Regards

Kapila 

Reply