Connection recovery

[fraburnham]

java.net.SocketException: Connection reset
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) ~[na:1.8.0_66-internal]
    at java.net.SocketOutputStream.write(SocketOutputStream.java:153) ~[na:1.8.0_66-internal]
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_66-internal]
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_66-internal]
    at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.8.0_66-internal]
    at com.rabbitmq.client.impl.SocketFrameHandler.flush(SocketFrameHandler.java:150) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.AMQConnection.flush(AMQConnection.java:518) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.AMQCommand.transmit(AMQCommand.java:125) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.AMQChannel.quiescingTransmit(AMQChannel.java:334) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:310) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:303) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.ChannelN.basicReject(ChannelN.java:1057) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.recovery.RecoveryAwareChannelN.basicReject(RecoveryAwareChannelN.java:72) ~[uberjar-standalone.jar:na]
    at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.basicReject(AutorecoveringChannel.java:362) ~[uberjar-standalone.jar:na]
    at langohr.basic$reject.invokePrim(basic.clj:214) ~[uberjar-standalone.jar:na]
    at com.farmlogs.conduit.subscription.ack_process$fn__17270.invoke(ack_process.clj:21) ~[uberjar-standalone.jar:na]
    at com.farmlogs.conduit.protocols$fn__7948$G__7943__7957.invoke(protocols.clj:3) ~[uberjar-standalone.jar:na]
    at com.farmlogs.conduit.subscription.ack_process$__GT_responder$fn__17274$fn__17279.invoke(ack_process.clj:32) ~[uberjar-standalone.jar:na]
    at com.farmlogs.conduit.subscription.ack_process$__GT_responder$fn__17274.invoke(ack_process.clj:32) [uberjar-standalone.jar:na]
    at clojure.core.async$thread_call$fn__5602.invoke(async.clj:434) [uberjar-standalone.jar:na]
    at clojure.lang.AFn.run(AFn.java:22) [uberjar-standalone.jar:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_66-internal]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_66-internal]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66-internal]

That is the first exception observed. After this point the queue is deleted from rmq (auto-delete true), work is lost and restarting the worker appears to be the only way to recover. The expectation is that rmq will recover without needed to restart the worker system.

[briprowe]

To be clear, recovery does work when auto-delete is false, correct?

[fraburnham]

For this worker no method of recovery seems to work as expected. I've seen this failure with auto-delete true and false. The userdata-update and catalog-field-lookup workers function as expected.