Handle request timeout more gracefully

[slominskir]

Currently if an async background data request (AJAX) timeout occurs (takes longer than 30 seconds) WAVE will not notify the user of the problem and instead displays a blank screen. The developer console will show the request was abandoned just after 30 seconds with no payload returned to the app data callback.

For example:

https://epicsweb.jlab.org/wave/?start=2022-01-01T07%3A50%3A00&end=2023-04-01T07%3A55%3A00&myaDeployment=history&myaLimit=100000&windowMinutes=30&title=&fullscreen=false&layoutMode=1&viewerMode=1&pv=R2XXITOT&R2XXITOTlabel=R2XXITOT&R2XXITOTcolor=%23a6cee3&R2XXITOTyAxisLabel=&R2XXITOTyAxisMin=&R2XXITOTyAxisMax=&R2XXITOTyAxisLog=&R2XXITOTscaler=

Results in:

and:

[slominskir]

More details:

The myquery log shows:

11-Apr-2023 10:14:03.910 SEVERE [https-jsse-nio-443-exec-34] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [IntervalController] in context with path [/myquery] threw exception
        jakarta.json.JsonException: I/O error while writing in JsonGenerator
                at org.eclipse.parsson.JsonGeneratorImpl.flushBuffer(JsonGeneratorImpl.java:652)
                at org.eclipse.parsson.JsonGeneratorImpl.writeString(JsonGeneratorImpl.java:605)
                at org.eclipse.parsson.JsonGeneratorImpl.writeString(JsonGeneratorImpl.java:611)
                at org.eclipse.parsson.JsonGeneratorImpl.write(JsonGeneratorImpl.java:214)
                at org.jlab.myquery.QueryController.writeFloatEvent(QueryController.java:55)
                at org.jlab.myquery.QueryController.generateFloatStream(QueryController.java:168)
                at org.jlab.myquery.IntervalController.doGet(IntervalController.java:258)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:705)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:814)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:223)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.jlab.myquery.CacheAndEncodingFilter.doFilter(CacheAndEncodingFilter.java:64)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:119)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
                at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:400)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:859)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1734)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.base/java.lang.Thread.run(Thread.java:833)
                Suppressed: jakarta.json.stream.JsonGenerationException: Generating incomplete JSON
                        at org.eclipse.parsson.JsonGeneratorImpl.close(JsonGeneratorImpl.java:521)
                        at org.jlab.myquery.IntervalController.doGet(IntervalController.java:204)
                        ... 27 more
        Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
                at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:349)
                at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:780)
                at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:685)
                at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:384)
                at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:362)
                at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96)
                at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:234)
                at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:304)
                at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
                at java.base/sun.nio.cs.StreamEncoder.write(StreamEncoder.java:132)
                at java.base/java.io.OutputStreamWriter.write(OutputStreamWriter.java:205)
                at org.eclipse.parsson.JsonGeneratorImpl.flushBuffer(JsonGeneratorImpl.java:648)
                ... 33 more

Or sometimes:

11-Apr-2023 10:19:01.118 SEVERE [https-jsse-nio-443-exec-46] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [IntervalController] in context with path [/myquery] threw exception
        jakarta.json.JsonException: I/O error while writing in JsonGenerator
                at org.eclipse.parsson.JsonGeneratorImpl.flushBuffer(JsonGeneratorImpl.java:652)
                at org.eclipse.parsson.JsonGeneratorImpl.writeString(JsonGeneratorImpl.java:605)
                at org.eclipse.parsson.JsonGeneratorImpl.writeString(JsonGeneratorImpl.java:611)
                at org.eclipse.parsson.JsonGeneratorImpl.write(JsonGeneratorImpl.java:214)
                at org.jlab.myquery.QueryController.writeFloatEvent(QueryController.java:55)
                at org.jlab.myquery.QueryController.generateFloatStream(QueryController.java:168)
                at org.jlab.myquery.IntervalController.doGet(IntervalController.java:258)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:705)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:814)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:223)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.jlab.myquery.CacheAndEncodingFilter.doFilter(CacheAndEncodingFilter.java:64)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:119)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
                at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:400)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:859)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1734)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.base/java.lang.Thread.run(Thread.java:833)
                Suppressed: jakarta.json.stream.JsonGenerationException: Generating incomplete JSON
                        at org.eclipse.parsson.JsonGeneratorImpl.close(JsonGeneratorImpl.java:521)
                        at org.jlab.myquery.IntervalController.doGet(IntervalController.java:204)
                        ... 27 more
        Caused by: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Connection reset by peer
                at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:349)
                at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:780)
                at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:685)
                at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:384)
                at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:362)
                at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96)
                at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:234)
                at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:304)
                at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
                at java.base/sun.nio.cs.StreamEncoder.write(StreamEncoder.java:132)
                at java.base/java.io.OutputStreamWriter.write(OutputStreamWriter.java:205)
                at org.eclipse.parsson.JsonGeneratorImpl.flushBuffer(JsonGeneratorImpl.java:648)
                ... 33 more
        Caused by: java.io.IOException: Connection reset by peer
                at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
                at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
                at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:132)
                at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:97)
                at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:53)
                at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:532)
                at org.apache.tomcat.util.net.SecureNioChannel.flush(SecureNioChannel.java:123)
                at org.apache.tomcat.util.net.SecureNioChannel.write(SecureNioChannel.java:795)
                at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1374)
                at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:764)
                at org.apache.tomcat.util.net.SocketWrapperBase.writeBlocking(SocketWrapperBase.java:589)
                at org.apache.tomcat.util.net.SocketWrapperBase.write(SocketWrapperBase.java:533)
                at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.doWrite(Http11OutputBuffer.java:547)
                at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:112)
                at org.apache.coyote.http11.Http11OutputBuffer.doWrite(Http11OutputBuffer.java:194)
                at org.apache.coyote.Response.doWrite(Response.java:625)
                at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:336)
                ... 44 more

[slominskir]

Related: JeffersonLab/myquery#1

Ideally myquery is streaming data, not building up huge response in memory and then sending all-at-once.