What is the expected outcome if failover happens while replay is in progress

rdesoju

Hi,
Consider a scenario where a Queue listener (JCSMP) is listening with CLIENT_ACK, typically a slow consumer processing the messages in the queue at a rate slower than producer in a guaranteed messaging approach with replay enabled.

What happens if failover is triggered while replay is in progress on a slow consumer which will typically have a queue depth and also replay messages being processed at a slower rate both at the same time.

Any thoughts?

Thanks

TomF

Hello @rdesoju, an interesting question. I haven't tried this myself, but I'll go it a go on Monday. Since PubSub+ replicates all state, including meta-data, on messages and queues, the backup node will know:

• that replay has been initiated on the queue;
• exactly where the consumer has got to in the replayed queue of messages.
  So, I am pretty confident the consumer will pick up where it left off.

Remember that replay re-sets the JSCMP flow state between the broker and the consumer, as the consumer will be in a different place in the queue once replay is initiated - for reasons like this. So actually, the consumer being slow doesn't really affect this case.

Now, if your question is: my consumer didn't catch up with the queue, then replay is initiated, what happens to the new messages that didn't get to the consumer before replay is initiated? Well, they will just appear in the queue of messages in order and will be delivered from the queue as replay completes - just like any other new message, say one queued while replay is still replaying old messages. Failover doesn't change any of this, because of the reasons above.

TomF

Hello @rdesoju, an interesting question. I haven't tried this myself, but I'll go it a go on Monday. Since PubSub+ replicates all state, including meta-data, on messages and queues, the backup node will know:

• that replay has been initiated on the queue;
• exactly where the consumer has got to in the replayed queue of messages.
  So, I am pretty confident the consumer will pick up where it left off.

Remember that replay re-sets the JSCMP flow state between the broker and the consumer, as the consumer will be in a different place in the queue once replay is initiated - for reasons like this. So actually, the consumer being slow doesn't really affect this case.

Now, if your question is: my consumer didn't catch up with the queue, then replay is initiated, what happens to the new messages that didn't get to the consumer before replay is initiated? Well, they will just appear in the queue of messages in order and will be delivered from the queue as replay completes - just like any other new message, say one queued while replay is still replaying old messages. Failover doesn't change any of this, because of the reasons above.

rdesoju

@TomF , Thanks for your insights on this.
Please share if you observe any deviation from above from your tests.

Thanks