Question

Background Jobs stuck in Posted not getting Executed

  • 13 December 2021
  • 15 replies
  • 1536 views

Userlevel 4
Badge +6

Hi,

Using Apps10 UPD8.

Background jobs are stuck in Posted status and not getting executed in any of Queues.

If I run Transaction_sys.Process_all_pending for a particular queue manually via plsql then it process the jobs in that queue but only for that time. So every time to execute the jobs in queue i have to run it manually for the queue I want to process.

Already checked below:

  1. Job Queue processes value is fine in database.
  2. CJQ0 backgorund process is running.
  3. Batch Queues are enabled. Done Init All Queues and individual batch queue.
  4. No jobs stuck in Executing State.
  5. Restarted Database and IFS MWS servers.

Issue is coming in Prod environment. Will appreciate the quick response.


This topic has been closed for comments

15 replies

Userlevel 5
Badge +12

Hi @sahilgupta ,

I think you have checked all the requirements except for the database processes checking. Check if they are scheduled correctly

 

 

Userlevel 7
Badge +21

Hi @sahilgupta,

Is the DB running Oracle 12.2? Probably a long shot but in a specific version in 12.2, i’ve come across a but that messed with the scheduler jobs.  i would have expected this to happen in the other environments as well, but no harm in checking. This was caused by a bundle patch.

As part of Bug 26962111, new views were created for the scheduler and the script to create the views is not being called when applying either of the Windows Bundle Patches identified.

The views that are to be created are

sys.scheduler$_job_refresh

sys.scheduler$_lwjob_refresh

sys.scheduler$_window_refresh

Prior to running the catschv.sql script a describe command will fail on the above 3 objects as the objects do not exist

Running the script, catschv.sql manually as the SYS user creates the necessary views to restore scheduler functionality.

Manually run the script by doing the following:

  1. Change directory to the ORACLE_HOME\rdbms\admin directory on your host machine
  2. Connect to the database as the SYS user or \ as sysdba
  3. Run the catschv.sql script

 

Cheers

Userlevel 4
Badge +6

Hi @sahilgupta ,

I think you have checked all the requirements except for the database processes checking. Check if they are scheduled correctly

 

 

Hi Ruchira,
Thanks for the reply. Database Process are also in SCHEDULED STATUS. But Next Date is not changing so it means they are not getting executed automatically.

 

 

Userlevel 4
Badge +6

Y

Userlevel 5
Badge +6

Hi @sahilgupta,

Would you try the following steps. If just doing the below steps does not resolve the issue, restart the database after the changes.

1.    Check no of job queue processes in the database
    Show parameter job;

2.    Alter job queue processes to 0 
    Alter system set job_queue_processes=0;

3.    Stop database processes
    EXEC Install_Tem_SYS.Disable_Scheduler_Processes;

4.    Alter job queue processes to it’s original value
    Alter system set job_queue_processes=XX;

5.    Start database processes
    EXEC Install_Tem_SYS.Enable_Scheduler_Processes;

 

Thanks,

Jagath

Userlevel 4
Badge +6

Hi @sahilgupta,

Would you try the following steps. If just doing the below steps does not resolve the issue, restart the database after the changes.

1.    Check no of job queue processes in the database
    Show parameter job;

2.    Alter job queue processes to 0 
    Alter system set job_queue_processes=0;

3.    Stop database processes
    EXEC Install_Tem_SYS.Disable_Scheduler_Processes;

4.    Alter job queue processes to it’s original value
    Alter system set job_queue_processes=XX;

5.    Start database processes
    EXEC Install_Tem_SYS.Enable_Scheduler_Processes;

 

Thanks,

Jagath

Hi Jagath,

Thanks, tried the given steps but still the same issue.

The issue seems to be with Database processes for batch queues. Even though the status is SCHEDULE they are not executing the jobs in queues. Also the time in Next Date for database processes is not changing.

Regards,

Sahil Gupta

Userlevel 5
Badge +12

Hi Again,

Just clearing errands here, do you have by any chance a fixed date set in the database?

Userlevel 4
Badge +6

Hi @sahilgupta,

Is the DB running Oracle 12.2? Probably a long shot but in a specific version in 12.2, i’ve come across a but that messed with the scheduler jobs.  i would have expected this to happen in the other environments as well, but no harm in checking. This was caused by a bundle patch.

As part of Bug 26962111, new views were created for the scheduler and the script to create the views is not being called when applying either of the Windows Bundle Patches identified.

The views that are to be created are

sys.scheduler$_job_refresh

sys.scheduler$_lwjob_refresh

sys.scheduler$_window_refresh

Prior to running the catschv.sql script a describe command will fail on the above 3 objects as the objects do not exist

Running the script, catschv.sql manually as the SYS user creates the necessary views to restore scheduler functionality.

Manually run the script by doing the following:

  1. Change directory to the ORACLE_HOME\rdbms\admin directory on your host machine
  2. Connect to the database as the SYS user or \ as sysdba
  3. Run the catschv.sql script

 

Cheers

Hi Sajith, Thanks for suggestion, Tried the same but issue is still there.

Userlevel 4
Badge +6

Hi Again,

Just clearing errands here, do you have by any chance a fixed date set in the database?

Hi Ruchira,

There is no FIXED_DATE set in database.

Regards,

Sahil Gupta

Userlevel 5
Badge +9

Hi @sahilgupta, Do you by any chance using a container database, so this is a PDB where you having this issue?

Userlevel 4
Badge +6

Hi @sahilgupta, Do you by any chance using a container database, so this is a PDB where you having this issue?

Hi Kasun. Its non-cdb database.

Userlevel 7
Badge +21

Hi @sahilgupta ,

Did you take a look at the alert log to see whether there is something funny going on in the back end? From the sound of it it does seem like the cjq process is having trouble spawning j0 threads so wondering whether you have any unusual errors in the alert log. 

Cheer

 

Userlevel 5
Badge +12

Hi @sahilgupta ,

Did you take a look at the alert log to see whether there is something funny going on in the back end? From the sound of it it does seem like the cjq process is having trouble spawning j0 threads so wondering whether you have any unusual errors in the alert log. 

Cheer

 

Agree with Sajith, has to be something with cjq process

this is what I could think of;

connect / as sysdba

select * from dba_objects where object_name='SCHEDULER_DISABLED';

if you get any rows for the above try (re)/enabling the scheduler

connect / as sysdba

exec dbms_scheduler.set_scheduler_attribute('SCHEDULER_DISABLED','FALSE');

 

Userlevel 4
Badge +6

Hi @Sajith D,

Yes I check the alert log file and there was nothing alarming or error in alert log file.
But the issue was resolved today morning after applying the delivery. I think the reason is because delivery stops the scheduler and background processes in the beginning and then starts them in the end so something which was not running got triggered because of the delivery deployment.

Thanks for your suggestions.

Regards,
Sahil Gupta

 

Userlevel 4
Badge +6

Hi @Ruchira ,

Yes I check the cjq processes and wasn’t able to find any issue with the same.

But the issue was resolved today morning after applying the delivery. I think the reason is because delivery stops the scheduler and background processes in the beginning and then starts them in the end so something which was not running got triggered because of the delivery deployment.

Thanks for your suggestions and time.

Regards,
Sahil Gupta