OData pods are frequently going down in the Production

Question

Hello,OData pods are frequently going down in both the Production and lower environments with following error:================================================================================  com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl$7.run(WSManagedConnectionFactoryImpl.java:1017)
	 at java.base@11.0.19/java.security.AccessController.doPrivileged(Native Method)
	 at java.base@11.0.19/java.security.AccessController.doPrivilegedWithCombiner(Unknown Source)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.getConnectionUsingDS(WSManagedConnectionFactoryImpl.java:1017)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.getConnection(WSManagedConnectionFactoryImpl.java:846)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.createManagedConnection(WSManagedConnectionFactoryImpl.java:696)
	 at com.ibm.ejs.j2c.FreePool.createManagedConnectionWithMCWrapper(FreePool.java:1373)
	 at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1247)
	 at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:1442)
	 at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:574)
	 at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:306)
	 at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:140)
	 at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:114)
	 at web.ODPReadinessCheck.call(ODPReadinessCheck.java:42)
	 at web.ODPReadinessCheck$Proxy$_$$_WeldClientProxy.call(Unknown Source)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20CDIBeanInvokerImpl.checkAllBeans(HealthCheck20CDIBeanInvokerImpl.java:80)
	 at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	 at java.base@11.0.19/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	 at java.base@11.0.19/java.lang.reflect.Method.invoke(Unknown Source)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:77)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:99)
	 at com.sun.proxy.$Proxy151.checkAllBeans(Unknown Source)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20ExecutorImpl.runHealthChecks(HealthCheck20ExecutorImpl.java:98)
	 at com.ibm.ws.microprofile.health20.internal.HealthCheck20ServiceImpl.performHealthCheck(HealthCheck20ServiceImpl.java:137)
	 at com.ibm.ws.microprofile.health20.internal.servlet.HealthCheckReadinessServlet.service(HealthCheckReadinessServlet.java:43)
	 at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:193)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:98)
	 at com.ibm.ws.app.manager.wab.internal.OsgiDirectoryProtectionFilter.doFilter(OsgiDirectoryProtectionFilter.java:90)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.security.jaspi.JaspiServletFilter.doFilter(JaspiServletFilter.java:56)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1002)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1140)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)
	 at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)
	 at com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)
	 at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)
	 at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:281)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1184)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:453)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:412)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:167)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:75)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:514)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:584)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:968)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1057)
	 at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	 at java.base@11.0.19/java.lang.Thread.run(Unknown Source)

The following table shows the events that have run during this request.

Duration      Operation
45426.424ms + websphere.servlet.service | health | HealthCheckReadinessServlet ","ibm_threadId":"00000038","_timestamp":"2025-07-07T07:01:39.670+0000","event.id":"TRAS0112W","module":"com.ibm.ws.request.timing.manager.SlowRequestManager","log.level":"WARNING","labels.ibm_sequence":"1751871699670_000000000009A","ext_requestID":"AAFbUsRjU5f_AAAAAAAAAeq","process.thread.name":"Default Executor-thread-19"}{"event.module":"liberty_message","host.name":"ifsapp-odata-68fc47dfd7-wcqd5","ibm_userDir":"/opt/ifs/wlp/usr/","ibm_serverName":"defaultServer","message":"TRAS0114W: Request AAFbUsRjU5f_AAAAAAAAAdt has been running on thread 00000033 for at least 618646.296ms. The following table shows the events that have run during this request.

Duration       Operation
618649.407ms + websphere.servlet.service | ifs-odata-provider | OData-Provider | /CInventStockAgeingRep.svc/VirtualOrderReports(Objkey='85D3CB3005A049F8AD56F6F5DE0D9A08')/IfsApp.CInventStockAgeingRep.VirtualOrderReport_OrderReport?odata-debug=json
     0.657ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {call Login_SYS.Init_ODP_Sys_Session_()}
     0.486ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
BEGIN
? := Fnd_Odata_Provider_Sys.Get_Projection_Meta_Version(?);
END;

0.415ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
BEGIN
Bpa_Sys.Init_Session;
END;
     2.439ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.783ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     0.420ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     0.244ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     0.184ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     2.044ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.354ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     0.375ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | SELECT A.* FROM (SELECT objkey objkey, 'W/"'||objkey||'"' etag FROM ORDER_REPORT_TEMPLATE_VIRTUAL_ORDER_REPORT_VRT WHERE ((obj_created_by = fnd_session_api.Get_Fnd_User) AND ((objkey = ?) AND obj_created_by = fnd_session_api.Get_Fnd_User))) A
     0.199ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     1.707ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.298ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     0.436ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
BEGIN
? := Bpa_Sys.Get_Events;
END;
     0.281ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
BEGIN
Bpa_Sys.Init_Session;
END;
     1.943ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.458ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     0.253ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
BEGIN
Bpa_Sys.Init_Session;
END;
     1.877ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.491ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
     0.324ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     0.148ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     0.131ms       websphere.datasource.psExecuteQuery | jdbc/fndbas-odp | select process_key, workflow_execution_db, workflow_timing_db, projection_name, action, entity_set_name, call_name, is_read_enabled, is_create_enabled, is_update_enabled, is_delete_enabled, OBJKEY   
from BPMN_PROJECTION
cross join (select ? as A, ? as B, ? as C, ? as D, ? as E,  ? as F from dual) D 
where  (A is null or A = projection_name) 
   AND (B is null or process_key = B) 
   AND (C is null or (C = 'R' and is_read_enabled = 'TRUE') or (C = 'C' and is_create_enabled = 'TRUE') or (C = 'U' and is_update_enabled = 'TRUE') or (C = 'D' and is_delete_enabled = 'TRUE')) 
   AND (is_enabled = 'TRUE') 
   AND (D is null or entity_set_name = D) 
   AND (E is null or call_name = E) 
   AND (F is null or workflow_timing_db = F)
     2.076ms       websphere.datasource.psExecute | jdbc/fndbas-odp | {? = call Login_SYS.Init_Odp_Session_(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
     1.374ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;
 59727.684ms       websphere.datasource.psExecute | jdbc/fndbas-odp | DECLARE
	return_array_ Order_Report_Template_SVC.Report_Result_Art;
	cursor_ SYS_REFCURSOR;
BEGIN
	return_array_ := Order_Report_Template_SVC.Do_Order_Report(?, virtual_order_report## => '');
	OPEN cursor_ FOR SELECT * FROM TABLE(return_array_);
	? := cursor_;
END;
322714.483ms       websphere.datasource.psExecute | jdbc/fndbas-odp | BEGIN Client_SYS.Get_Trace(?); END;","ibm_threadId":"00000083","_timestamp":"2025-07-07T07:01:13.868+0000","event.id":"TRAS0114W","module":"com.ibm.ws.request.timing.manager.HungRequestManager","log.level":"WARNING","labels.ibm_sequence":"1751871673868_000000000009B","process.thread.name":"Default Executor-thread-40"}{"event.module":"liberty_message","host.name":"ifsapp-odata-68fc47dfd7-wcqd5","ibm_userDir":"/opt/ifs/wlp/usr/","ibm_serverName":"defaultServer","message":"TRAS0112W: Request AAFbUsRjU5f_AAAAAAAAAfO has been running on thread 00000027 for at least 22314.738ms. The following stack trace shows what this thread is currently running.

at org.jboss.weld.contexts.CreationalContextImpl.<init>(CreationalContextImpl.java:72)
	 at org.jboss.weld.manager.BeanManagerImpl.createCreationalContext(BeanManagerImpl.java:1121)
	 at org.jboss.weld.manager.BeanManagerImpl.createCreationalContext(BeanManagerImpl.java:201)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20CDIBeanInvokerImpl.getHealthCheckBeans(HealthCheck20CDIBeanInvokerImpl.java:137)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20CDIBeanInvokerImpl.getHealthCheckReadinessBeans(HealthCheck20CDIBeanInvokerImpl.java:105)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20CDIBeanInvokerImpl.checkAllBeans(HealthCheck20CDIBeanInvokerImpl.java:71)
	 at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	 at java.base@11.0.19/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	 at java.base@11.0.19/java.lang.reflect.Method.invoke(Unknown Source)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:77)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:99)
	 at com.sun.proxy.$Proxy151.checkAllBeans(Unknown Source)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20ExecutorImpl.runHealthChecks(HealthCheck20ExecutorImpl.java:98)
	 at com.ibm.ws.microprofile.health20.internal.HealthCheck20ServiceImpl.performHealthCheck(HealthCheck20ServiceImpl.java:137)
	 at com.ibm.ws.microprofile.health20.internal.servlet.HealthCheckReadinessServlet.service(HealthCheckReadinessServlet.java:43)
	 at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:193)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:98)
	 at com.ibm.ws.app.manager.wab.internal.OsgiDirectoryProtectionFilter.doFilter(OsgiDirectoryProtectionFilter.java:90)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.security.jaspi.JaspiServletFilter.doFilter(JaspiServletFilter.java:56)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1002)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1140)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)
	 at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)
	 at com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)
	 at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)
	 at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:281)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1184)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:453)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:412)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:167)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:75)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:514)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:584)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:968)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1057)
	 at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	 at java.base@11.0.19/java.lang.Thread.run(Unknown Source)

The following table shows the events that have run during this request.

Duration      Operation
22533.825ms + websphere.servlet.service | health | HealthCheckReadinessServlet ","ibm_threadId":"00000082","_timestamp":"2025-07-07T07:00:28.607+0000","event.id":"TRAS0112W","module":"com.ibm.ws.request.timing.manager.SlowRequestManager","log.level":"WARNING","labels.ibm_sequence":"1751871628607_0000000000098","process.thread.name":"Default Executor-thread-39"}{"event.module":"liberty_message","host.name":"ifsapp-odata-68fc47dfd7-wcqd5","ibm_userDir":"/opt/ifs/wlp/usr/","ibm_serverName":"defaultServer","message":"TRAS0112W: Request AAFbUsRjU5f_AAAAAAAAAfS has been running on thread 00000081 for at least 21849.090ms. The following stack trace shows what this thread is currently running.

at java.base@11.0.19/jdk.internal.misc.Unsafe.park(Native Method)
	 at java.base@11.0.19/java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
	 at java.base@11.0.19/java.util.concurrent.FutureTask.awaitDone(Unknown Source)
	 at java.base@11.0.19/java.util.concurrent.FutureTask.get(Unknown Source)
	 at oracle.ucp.util.UCPTaskManagerImpl$5.get(UCPTaskManagerImpl.java:261)
	 at oracle.ucp.common.UniversalPooledConnectionImpl.isValidHelper(UniversalPooledConnectionImpl.java:360)
	 at oracle.ucp.common.UniversalPooledConnectionImpl.isValid(UniversalPooledConnectionImpl.java:322)
	 at oracle.ucp.common.FailoverableUniversalPooledConnectionBase.isValid(FailoverableUniversalPooledConnectionBase.java:17)
	 at oracle.ucp.jdbc.oracle.OracleUniversalPooledConnection.isValid(OracleUniversalPooledConnection.java:242)
	 at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnectionAndValidateHelper(UniversalConnectionPoolImpl.java:201)
	 at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnectionAndValidate(UniversalConnectionPoolImpl.java:153)
	 at oracle.ucp.common.UniversalConnectionPoolImpl.borrowConnection(UniversalConnectionPoolImpl.java:126)
	 at oracle.ucp.jdbc.JDBCConnectionPool.borrowConnection(JDBCConnectionPool.java:174)
	 at oracle.ucp.jdbc.oracle.OracleJDBCConnectionPool.borrowConnection(OracleJDBCConnectionPool.java:624)
	 at oracle.ucp.jdbc.oracle.OracleConnectionConnectionPool.borrowConnection(OracleConnectionConnectionPool.java:106)
	 at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:2036)
	 at oracle.ucp.jdbc.PoolDataSourceImpl.access$500(PoolDataSourceImpl.java:207)
	 at oracle.ucp.jdbc.PoolDataSourceImpl$31.build(PoolDataSourceImpl.java:4367)
	 at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:1970)
	 at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:1933)
	 at oracle.ucp.jdbc.PoolDataSourceImpl.getConnection(PoolDataSourceImpl.java:1918)
	 at com.ibm.ws.rsadapter.impl.DatabaseHelper.getConnectionFromDatasource(DatabaseHelper.java:1465)
	 at com.ibm.ws.rsadapter.impl.OracleHelper.getConnectionFromDatasource(OracleHelper.java:864)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl$7.run(WSManagedConnectionFactoryImpl.java:1030)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl$7.run(WSManagedConnectionFactoryImpl.java:1017)
	 at java.base@11.0.19/java.security.AccessController.doPrivileged(Native Method)
	 at java.base@11.0.19/java.security.AccessController.doPrivilegedWithCombiner(Unknown Source)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.getConnectionUsingDS(WSManagedConnectionFactoryImpl.java:1017)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.getConnection(WSManagedConnectionFactoryImpl.java:846)
	 at com.ibm.ws.rsadapter.impl.WSManagedConnectionFactoryImpl.createManagedConnection(WSManagedConnectionFactoryImpl.java:696)
	 at com.ibm.ejs.j2c.FreePool.createManagedConnectionWithMCWrapper(FreePool.java:1373)
	 at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1247)
	 at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:1442)
	 at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:574)
	 at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:306)
	 at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:140)
	 at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:114)
	 at web.ODPReadinessCheck.call(ODPReadinessCheck.java:42)
	 at web.ODPReadinessCheck$Proxy$_$$_WeldClientProxy.call(Unknown Source)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20CDIBeanInvokerImpl.checkAllBeans(HealthCheck20CDIBeanInvokerImpl.java:80)
	 at jdk.internal.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	 at java.base@11.0.19/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	 at java.base@11.0.19/java.lang.reflect.Method.invoke(Unknown Source)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:77)
	 at com.ibm.ws.context.service.serializable.ContextualInvocationHandler.invoke(ContextualInvocationHandler.java:99)
	 at com.sun.proxy.$Proxy151.checkAllBeans(Unknown Source)
	 at com.ibm.ws.microprofile.health20.services.impl.HealthCheck20ExecutorImpl.runHealthChecks(HealthCheck20ExecutorImpl.java:98)
	 at com.ibm.ws.microprofile.health20.internal.HealthCheck20ServiceImpl.performHealthCheck(HealthCheck20ServiceImpl.java:137)
	 at com.ibm.ws.microprofile.health20.internal.servlet.HealthCheckReadinessServlet.service(HealthCheckReadinessServlet.java:43)
	 at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)
	 at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:193)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:98)
	 at com.ibm.ws.app.manager.wab.internal.OsgiDirectoryProtectionFilter.doFilter(OsgiDirectoryProtectionFilter.java:90)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.security.jaspi.JaspiServletFilter.doFilter(JaspiServletFilter.java:56)
	 at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1002)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1140)
	 at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)
	 at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)
	 at com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)
	 at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)
	 at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:281)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1184)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:453)
	 at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:412)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
	 at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:167)
	 at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:75)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:514)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:584)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:968)
	 at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1057)
	 at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	 at java.base@11.0.19/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	 at java.base@11.0.19/java.lang.Thread.run(Unknown Source)

The following table shows the events that have run during this request.

Duration      Operation
45426.439ms + websphere.servlet.service | health | HealthCheckReadinessServlet ","ibm_threadId":"00000070","_timestamp":"2025-07-07T07:01:39.670+0000","event.id":"TRAS0112W","module":"com.ibm.ws.request.timing.manager.SlowRequestManager","log.level":"WARNING","labels.ibm_sequence":"1751871699670_0000000000099","ext_requestID":"AAFbUsRjU5f_AAAAAAAAAej","process.thread.name":"Default Executor-thread-21"}*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message can't create name string at ../src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 827*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message can't create name string at ../src/java.instrument/share/native/libinstrument/JPLISAgent.c line: 827

maheshmuz · Answer

We’ve noticed improved stability and performance when running two OData pods instead of just one.
If you are currently running a single pod for ifsapp-odata, consider scaling the deployment to 2 replicas.

Did you find what you're looking for? If not:

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded