Question

BPA Workflow execution stuck in In Progress (IP) state?

  • 13 November 2023
  • 9 replies
  • 164 views

Userlevel 2
Badge +8

Recently workflows sometimes get stuck in ‘IP’ (In Progress) state, and never reach state ‘DONE’, it seems:

 

They are mostly started from background jobs.

Some of them are (not on the image above) more than one month old, still stuck.

The same workflow (finUpdateCurrencyRatesForCurrencyTask) always gets executed properly when started online.

What can be done in a situation like this?

 

version: 23.1.6


9 replies

Badge

In BPA_ASYNC_SYS_TAB we have all the asynchronous job executions with the status. When there is a failure in the execution, we are retrying the execution another 2 time (altogether 3) by keeping the status In Progress.(Till RETRY_COUNT 3) 

From 23R2 we have the option of capturing observation for asynchronous executions as well and use these observations directly to troubleshooting purposes.

For the timing being you can use the “ATIR” data as the input data for troubleshooting (debug) and compare the execution for “IP” status and “DONE”.  (For this you need to convert the ATIR data to valid JSON)

Userlevel 2
Badge +8

Thanks for info. 
Unfortunately I did not get much closer to the solution as the workflow apparently runs fine when in debugging mode.

Anyway, I deployed this workflow to another system it works well there!

So it seems something somehow is stuck regarding this workflow in the original system.

 

So moving forward I have another problem now:
How to get notified - at user level - of errors in the workflow execution when run in background?

This workflow will primarily run in the background, started by a background job.

I deliberately put some error in the workflow and I get no notification of the error.

The background job that starts it gets to ‘Executed’ state without any errors or warnings.

What is the solution here, how can one see when the workflow execution in the background runs into an exception?

 

 

 

Badge +4

Thanks for info. 
Unfortunately I did not get much closer to the solution as the workflow apparently runs fine when in debugging mode.

Anyway, I deployed this workflow to another system it works well there!

So it seems something somehow is stuck regarding this workflow in the original system.

 

So moving forward I have another problem now:
How to get notified - at user level - of errors in the workflow execution when run in background?

This workflow will primarily run in the background, started by a background job.

I deliberately put some error in the workflow and I get no notification of the error.

The background job that starts it gets to ‘Executed’ state without any errors or warnings.

What is the solution here, how can one see when the workflow execution in the background runs into an exception?


I also want to know how to monitor the async flows. BPA_ASYNC_SYS_TAB contains the execution log for 3 months, but I can’t find a view or projection/page that contains this data. I think It’s unsafe to run a Workflow as async when failures will not be monitored. For now I run them as After.

@jasahu , how did you start a Workflow from a background job?

Userlevel 2
Badge +8

Hi @StartStop,

 

It is done as it is described here:

Currency Rate Update (ACCRUL) - Technical Documentation For IFS Cloud

Userlevel 4
Badge +9

@yohan  @jasahu  did you find a solution for this problem? 
We are struggling to understand why workflows are getting stuck in IP state.

 

How did you do the retry option in the workflow?

Userlevel 2
Badge +8

@yohan  @jasahu  did you find a solution for this problem? 
We are struggling to understand why workflows are getting stuck in IP state.

 

How did you do the retry option in the workflow?

Hi,

Unfortunately we could not find a solution for this ‘getting stuck’ thing.
So we decided stop using workflows for background style processing as this problem seriously hinders usability and went back to conventional PLSQL solutions.
It is sad because we put a lot of work in it, but eventually came to the conclusion that BPA in background is not ready for production use in general.

Badge

Hi @jasahu  @kvbe ,

Are there any cascade workflows associated with this asynchronous workflow execution?

If there are errors related to executing the Workflow they will be logged in oData logs. Please see if any errors logged there. In addition, starting from 24R1, we are adding error details to the Async table if there are any errors during execution. Additionally, from 23R2 onwards, we have implemented asynchronous observation capture for troubleshooting purposes. These features can also be used for troubleshooting Async Workflows.

Async executions do not provide immediate notifications to users in case of errors. If none of the methods are successful in locating the error, please raise a ticket to IFS Support for further investigations.

Thank you, 

Yohan

Userlevel 4
Badge +9

@yohan can you see the odata logs on an IFS Cloud hosted setup? Or only on a remote deployment?

Userlevel 2
Badge +8

Hi @jasahu  @kvbe ,

Are there any cascade workflows associated with this asynchronous workflow execution?

If there are errors related to executing the Workflow they will be logged in oData logs. Please see if any errors logged there. In addition, starting from 24R1, we are adding error details to the Async table if there are any errors during execution. Additionally, from 23R2 onwards, we have implemented asynchronous observation capture for troubleshooting purposes. These features can also be used for troubleshooting Async Workflows.

Async executions do not provide immediate notifications to users in case of errors. If none of the methods are successful in locating the error, please raise a ticket to IFS Support for further investigations.

Thank you, 

Yohan

Hi @yohan,

 

No cascading workflows are involved.

Thanks for all these improvements!

Reply