TTLs & Message Processing

karthikv
karthikv Member Posts: 6 ✭
in PubSub+ Event Broker #1

Dear Experts,

I am trying to use JCSMP client and I have three questions w.r.t message processing at client side and TTL :

  • I have realised that if a message expires just at the moment it is being listened to at the client side, the message would be processed by client and also re-delivered via DMQ (as I am listening to DMQ as well). What is the general recommendation to avoid this re-processing.
  • The same would also happen if the processing is taking really longer meanwhile the message expires and it is send to DMQ. Is there anyway to avoid sending such messages being processed to DMQ ?
  • Is there any way to reject the expired messages by configuration at the broker level itself, and not by comparing the expiration or TTL. I understand such a configuration is available with Apache QPID, but atleast I could not find anything related in the JCSMP API documentation. Kindly point me out if there are any such configuration available

Thanks & Best Regards,
Karthik

Tagged:

Best Answer

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #2 Answer ✓

    Hi there @karthikv,

    Right, ok… for your first point (and your 2nd point too): yes the broker will move a TTL-expired message to the queue's DMQ (if configured) as soon as it expires. ("as soon" == broker checks for TTL expiry once-per-second, that's the maximum resolution, but if you have a TON of messages on your queue then the broker might be a bit slower at expiring messages to DMQ). If the message is in-flight or at the client but hasn't been ACKed yet, doesn't matter… it will be moved to DMQ. There's nothing you can do in the client or the broker to prevent this. (today)

    Strategies to help minimize this:

    • Increase TTL of message / queue
    • Process messages faster! 😅
    • Have the publisher set the "expiration" on the message as well to give consuming clients an indication that the message might have already moved to DMQ. The broker doesn't action on the expiration, just the TTL.
    • You can always have unintentionally slow consumers, bugs happen, there will always be some corner-case where you will end up in this situation even with the best strategies. It's always best to code your consumers to be able to detect/handle duplicate deliveries.
    • That said: the consumers listening to the DMQ should probably be very different apps than the regular consumers on the main queue… they'd be more error-handling, exception-dealing-with applications… so "duplicate processing" shouldn't really be an issue.

    Note that even without a DMQ, using a regular exclusive queue, there are ways for the primary consumer to lose connection to the broker where it doesn't notice right away (hung socket through a proxy for example) and the app doesn't know it has been disconnected, and all unACKed messages are sent to the next standby consumer on the queue, while the first app is unaware and just keeps processing (nothing to do with TTL or expiration). At least in this case, the broker sets the "redelivered flag" to give an indication to the next consumer that it might be a dupe, but if they process it first, and then the slow original consumer finally processes the same message..!? Yup, handling dupes is important.

    I'm curious about QPID's behaviour. Is it just that: if a message has been sent to a consumer, it won't/can't expire to DMQ? The broker will hold it in the queue until either the consumer ACKs/NACKs or disconnects?

Answers

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #3 Answer ✓

    Hi there @karthikv,

    Right, ok… for your first point (and your 2nd point too): yes the broker will move a TTL-expired message to the queue's DMQ (if configured) as soon as it expires. ("as soon" == broker checks for TTL expiry once-per-second, that's the maximum resolution, but if you have a TON of messages on your queue then the broker might be a bit slower at expiring messages to DMQ). If the message is in-flight or at the client but hasn't been ACKed yet, doesn't matter… it will be moved to DMQ. There's nothing you can do in the client or the broker to prevent this. (today)

    Strategies to help minimize this:

    • Increase TTL of message / queue
    • Process messages faster! 😅
    • Have the publisher set the "expiration" on the message as well to give consuming clients an indication that the message might have already moved to DMQ. The broker doesn't action on the expiration, just the TTL.
    • You can always have unintentionally slow consumers, bugs happen, there will always be some corner-case where you will end up in this situation even with the best strategies. It's always best to code your consumers to be able to detect/handle duplicate deliveries.
    • That said: the consumers listening to the DMQ should probably be very different apps than the regular consumers on the main queue… they'd be more error-handling, exception-dealing-with applications… so "duplicate processing" shouldn't really be an issue.

    Note that even without a DMQ, using a regular exclusive queue, there are ways for the primary consumer to lose connection to the broker where it doesn't notice right away (hung socket through a proxy for example) and the app doesn't know it has been disconnected, and all unACKed messages are sent to the next standby consumer on the queue, while the first app is unaware and just keeps processing (nothing to do with TTL or expiration). At least in this case, the broker sets the "redelivered flag" to give an indication to the next consumer that it might be a dupe, but if they process it first, and then the slow original consumer finally processes the same message..!? Yup, handling dupes is important.

    I'm curious about QPID's behaviour. Is it just that: if a message has been sent to a consumer, it won't/can't expire to DMQ? The broker will hold it in the queue until either the consumer ACKs/NACKs or disconnects?

  • karthikv
    karthikv Member Posts: 6 ✭
    #4

    Thanks @Aaron for the detailed explanation. I was also thinking about exact same solution to either have a longer timeout or have the messages persisted for sometime on the client and use it to reject the message if it comes back via DMQ.

    This is because unfortunately we have a situation where we have to have the DMQ being listened by another application which again routes the same message to the queue for re-try scenarios and to have control over the retry and other features.

    The behaviour on QPID is the same w.r.t dispatching it to DMQ, but what differs is that it can filter out messages which are expired during transit by enabling the property jms.localMessageExpiry. This way the expired message is not consumed by the application and the only mechanism is that it would be consumed by the application bound to DMQ. I did not see such capability with JCSMP. Hope this clarifies what I meant, sorry that I was not clear earlier.

    Best Regards,
    Karthik

Categories

This Month's Leaders

This Week's Leaders