Question

Restarting print job due to rendering container failure

  • 28 April 2023
  • 2 replies
  • 328 views
P
Rank
Userlevel 2
Badge +5

None of the operational reports are working in one of my customer’s user place environments. They are on IFS Cloud 21R2 SU10. On premise customer. They have many other environments but the issue surfaced only in one of the environments. I would really appreciate it if you can give me some pointers to debug this issue.

Here are the things I have checked so far. 

  1. Verify  Message Queue  and Report Messages Queues are setup properly.
  2. Print Task Template is pointing to BATCH2
  3. Report Print Task Template is set up properly
  4. cleared the tables report_batch_pr_queue_tab and report_application_message_tab

Observation

Report Application messages are keep running for the same report.

I see the error below in the ifsapp-reporting-ren container log

{"type":"liberty_message","host":"ifsapp-reporting-ren-8bfb468cb-xwgkd","ibm_userDir":"\/opt\/ifs\/wlp\/usr\/","ibm_serverName":"defaultServer","message":"4735 ERROR [Framework]{ifsapp-reporting-ren} org.apache.fop.apps.Driver.renderFo(): Throwable catched during rendering process [Error GUID:2dd0191b-78c6-481e-a90e-8cca5b162ba5]\njava.lang.NullPointerException\n\tat org.apache.fop.render.pdf.fonts.LazyFont.getAscender(LazyFont.java:127)\n\tat org.apache.fop.layout.FontState.getAscender(FontState.java:270)\n\tat org.apache.fop.layout.LineArea.<init>(LineArea.java:843)\n\tat org.apache.fop.layout.BlockArea.getCurrentLineArea(BlockArea.java:313)\n\tat org.apache.fop.fo.FOText.addRealText(FOText.java:485)\n\tat org.apache.fop.fo.FOText.addText(FOText.java:474)\n\tat org.apache.fop.fo.FOText.layout(FOText.java:383)\n\tat org.apache.fop.fo.flow.Block.layout(Block.java:450)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat
org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat
org.apache.fop.fo.flow.Table.layout(Table.java:612)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat
org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat
org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat
org.apache.fop.fo.flow.Table.layout(Table.java:661)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat org.apache.fop.fo.flow.Table.layout(Table.java:661)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat org.apache.fop.fo.flow.Table.layout(Table.java:661)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat org.apache.fop.fo.flow.Table.layout(Table.java:661)\n\tat org.apache.fop.fo.flow.TableCell.layout(TableCell.java:545)\n\tat org.apache.fop.fo.flow.TableRow.layout(TableRow.java:693)\n\tat org.apache.fop.fo.flow.AbstractTableBody.layout(AbstractTableBody.java:380)\n\tat org.apache.fop.fo.flow.Table.layout(Table.java:661)\n\tat org.apache.fop.fo.flow.AbstractFlow.layout(AbstractFlow.java:281)\n\tat org.apache.fop.fo.flow.AbstractFlow.layout(AbstractFlow.java:237)\n\tat org.apache.fop.fo.pagination.PageSequence.makePage(PageSequence.java:555)\n\tat org.apache.fop.fo.pagination.PageSequence.format(PageSequence.java:484)\n\tat org.apache.fop.apps.StreamRenderer.render(StreamRenderer.java:265)\n\tat org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:233)\n\tat org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)\n\tat org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)\n\tat org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)\n\tat org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)\n\tat org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)\n\tat org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)\n\tat org.apache.xerces.parsers.XMLParser.parse(Unknown Source)\n\tat org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)\n\tat org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)\n\tat org.apache.fop.apps.Driver.render(Driver.java:634)\n\tat org.apache.fop.apps.Driver.render(Driver.java:619)\n\tat org.apache.fop.apps.Driver.run(Driver.java:730)\n\tat org.apache.fop.apps.Driver.renderFo(Driver.java:774)\n\tat ifs.application.reporting.rendering.PdfServices.renderFo(PdfServices.java:392)\n\tat ifs.application.reporting.rendering.PdfServices.renderPdfAfterTrasformation(PdfServices.java:102)\n\tat ifs.application.reporting.rendering.RenderingService.generatePdf(RenderingService.java:283)\n\tat ifs.application.reporting.rendering.RenderingService.generateReport(RenderingService.java:259)\n\tat ifs.application.reporting.rendering.RenderingService.getResult(RenderingService.java:173)\n\tat ifs.application.reportingservices.rendering.RenderingService.generatePdfFromPrintJob(RenderingService.java:180)\n\tat jdk.internal.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)\n\tat java.base\/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)\n\tat java.base\/java.lang.reflect.Method.invoke(Unknown Source)\n\tat com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:656)\n\tat com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:160)\n\tat org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:101)\n\tat com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:273)\n\tat org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:213)\n\tat com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:444)\n\tat org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:112)\n\tat org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)\n\tat org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)\n\tat org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)\n\tat org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)\n\tat org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:277)\n\tat com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:137)\n\tat com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:146)\n\tat com.ibm.websphere.jaxrs.server.IBMRestServlet.doPost(IBMRestServlet.java:104)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:706)\n\tat com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:96)\n\tat com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1258)\n\tat com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:746)\n\tat com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:443)\n\tat com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1227)\n\tat com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1011)\n\tat com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:75)\n\tat com.ibm.ws.webcontainer40.servlet.CacheServletWrapper40.handleRequest(CacheServletWrapper40.java:85)\n\tat com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:938)\n\tat com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:281)\n\tat com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1184)\n\tat com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:453)\n\tat com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:412)\n\tat com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)\n\tat com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)\n\tat com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)\n\tat com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)\n\tat com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:167)\n\tat com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:75)\n\tat com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)\n\tat com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)\n\tat com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:958)\n\tat com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1047)\n\tat com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base\/java.lang.Thread.run(Unknown Source)","ibm_threadId":"0000127f","ibm_datetime":"2023-04-25T14:46:57.192+0000","module":"SystemOut","loglevel":"SystemOut","ibm_methodName":"","ibm_className":"","ibm_sequence":"1682434017192_0000000027BE3","ext_requestID":"AARQOe2ODcO_AAAAAAAAJb\/","ext_thread":"Default Executor-thread-79"}

2 replies

hardik
Rank
Userlevel 3
Badge +7

Hello,

Yesterday i faced the same issue with one of the customer on IFSCLOUD.
Issue:- few print jobs in error state were getting retriggered continuously and causing issues for the new jobs also for the rendering pod.
This caused rendering pods getting auto restarted several times and that is when the email, print etc… many services got stuck and user started complaining.

But after rendering pod restart manually, and also killing few days older error out jobs in print manager, system is stable for now but still under monitoring.

Regards,
Hardik

B
Rank
Userlevel 5
Badge +15

Its sad to see a post open for 7+ months, has over 200 views, and so far IFS hasn’t commented on this once.

Reply


Terms & ConditionsCookie settings