Hello,
I'm trying to enable incremental refresh for FACT_GL_TRANSACTION, but I need to complete the initial refresh first. The dataset is quite large—refreshing accounting year (a full refresh using a criteria) 2024 alone takes around 15 hours. However, I'm unable to refresh the entire dataset. Once 2024 is fully refreshed the entity info says its no FRESH, attempting to refresh 2023 or any other year with entity criteria does not work. Any idea why?
Solved
FACT_GL_TRANSACTION Incremental Loading
+18Best answer by subashfestus
When using the criteria refresh to do the initial load, you have to do that using the “Refresh Using Criteria” command and cannot be used as a scheduled background job.
How the framework behaves in the situation is,
When we set the criteria to the fact entity and run the refresh as scheduled job the refresh takes place as an incremental load considering the last load dates, modified dates of source data with the criteria appended as a WHERE clause to filter the loaded data set. Then after the load it goes to fresh state because the incremental load is successful and the data loading is done.
But actually in this case what we need to do is set of conditional loads without considering the last load dates and modified dates of source data. The pure conditional load behavior is achieved through “Refresh Using Criteria” command.
You can check the incremental load logs to get an idea of how these 2 cases behaves differently.
There is a limitation with the “Refresh Using Criteria” command. Since it is invoked from the client end as an end user action it is subject to time out depending on the odata timeout setting. I think by default it is 10 minutes. So the conditional load job you are going to execute should finish before the timeout.
In your case you will have to go to “Accounting Year” + “Accounting period (month)” level or even smaller than that to reduce the amount of data loaded in a single run.
Best regards,
Subash
When using the criteria refresh to do the initial load, you have to do that using the “Refresh Using Criteria” command and cannot be used as a scheduled background job.
How the framework behaves in the situation is,
When we set the criteria to the fact entity and run the refresh as scheduled job the refresh takes place as an incremental load considering the last load dates, modified dates of source data with the criteria appended as a WHERE clause to filter the loaded data set. Then after the load it goes to fresh state because the incremental load is successful and the data loading is done.
But actually in this case what we need to do is set of conditional loads without considering the last load dates and modified dates of source data. The pure conditional load behavior is achieved through “Refresh Using Criteria” command.
You can check the incremental load logs to get an idea of how these 2 cases behaves differently.
There is a limitation with the “Refresh Using Criteria” command. Since it is invoked from the client end as an end user action it is subject to time out depending on the odata timeout setting. I think by default it is 10 minutes. So the conditional load job you are going to execute should finish before the timeout.
In your case you will have to go to “Accounting Year” + “Accounting period (month)” level or even smaller than that to reduce the amount of data loaded in a single run.
Best regards,
Subash
+18Hi
Thanks for the reply as always!
Your explanation makes sense. However, if i run the Refresh Using Criteria for FACT_GL_TRANSACTION using below criteria, it never completes. For this criteria i only have like 12 records to transfer.
Hi
What do you mean by “never completes”?
Enable the “Incremental load - log” and see how far the process goes by examining the logs.
Since this is a conditional/partial load you will not get a “Fresh” state.
Make sure that this Fact Entity is not in a “Refresh in Progress” state when you start the load.
BR,
Subash
1 person likes this
+18Hi
One of the related CMVs for FACT_GL_TRANSACTION was in the state unusable. Refresh using criteria is now working without any issues after activating it.
Thanks a lot for your help on this!
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.