Receive all unacknowledged messages using the .NET API
Are there any best practice recommendations to get all unacknowledged messages in a queue? The scenario I am thinking of is when messages are received by a client, but could not processed for what ever reason. The next time a new message is received on the queue, all the messages that are on the queue should be delivered to the connected client on FIFO basis.
The only I can think of this using the .NET API is to stop and start the flow each time a message is received, causing all messages to be redelivered.
Is there a better way?
Best Answer
-
@chewymints how do you know when you haven't received a message from the downstream application? Is the connection lost, a timeout, or a NACK?
Regardless, from the broker point of view, your application has received it. Your application should keep a hold (i.e. not dispose of the message) until you get the ack from your downstream application. If the downstream application doesn't receive the message, rather than asking Solace to send it again, re-send it in your application. Your application should manage the interaction with the downstream application. Otherwise you are asking the broker to understand the state of your downstream application - which it doesn't know about.
This makes managing failure and restarts, far easier, since you know exactly which application is responsible for what state.
5
Answers
-
Chewy Mints! Nice name. Ok, so
start()andstop()on a Flow doesn't actually close it... it's used for flow-control (pardon the overloading of the term). And when you callstop()there could still be an AD-Window's worth amount of messages in-flight from the broker, or precached by the API. Messages will not be made available for redelivery until you actually close the Flow (i.e. unbind from the queue).Also yes, Solace is always FIFO. For persistent queues, as well as via Direct messaging.
So if I understand your question correctly, you're saying you have a .NET SMF API client that's receiving a bunch of messages from a queue, has a problem with one of them (or some of them), and you want these messages to be redelivered? To someone else, or to the same client? Or you don't want the problem messages to be redelivered?
I'm having a bit of a hard time understanding exactly what is your question and/or what you want to happen. Can you spell it out a bit more for me? Thanks!
1 -
@Aaron: Thanks for the reply.
I have a .NET client that is subscribing to a queue and processing them and finally calling other downstream clients based on the content of the message. The message is only acknowledged if the downstream clients are available.If a downstream client is not available, the application will not ACK the message. However, the next time any other message is sent on the queue, I want the earlier message to be delivered first, followed by the new messages to the same client.
Does this make sense? I might be using the wrong terminology - I am pretty new to Solace.
0 -
@chewymints how do you know when you haven't received a message from the downstream application? Is the connection lost, a timeout, or a NACK?
Regardless, from the broker point of view, your application has received it. Your application should keep a hold (i.e. not dispose of the message) until you get the ack from your downstream application. If the downstream application doesn't receive the message, rather than asking Solace to send it again, re-send it in your application. Your application should manage the interaction with the downstream application. Otherwise you are asking the broker to understand the state of your downstream application - which it doesn't know about.
This makes managing failure and restarts, far easier, since you know exactly which application is responsible for what state.
5 -
@TomF : I have considered that option - thank you for reiterating the concept.
The only issue I have with that is the application will need to maintain the state of all faulting messages. This seems to be something Solace can do out of the box.
Thanks again for the help.0 -
Glad Tom's answer helped there! Yeah, the broker will not resend any message that it has already sent, as long as the application's connection to the queue is still up/valid. So the broker will send you the new message when it arrives, as it came after the previous message (that your downstream application had a problem with). If you want to NACK that problem message back onto the queue, and maintain order, then you need to close/unbind the Flow to the queue, and then all unacknowledged messages in your application will get put back on the queue and made available for redelivery to another application. (or the same one, when it rebinds).
Note again, if order is really important (FIFO) then you need to use an Exclusive queue. If using a Non-Exclusive queue for load-balancing among parallel applications, if one application NACKs a message back onto the queue, that message will now be "behind" messages that were previously sent to another application. If that makes sense!
0 -
@chewymints about "the application will need to maintain state..." Yes, but only in memory - it doesn't need to persist this state (as far as which message is been successfully processed) which is exactly what we're trying to achieve.
Imagine you have 1 message that's been processed, one that you've received but not sent on yet, and one that you've sent to your downstream app but don't have an ack for.
The one you've processed is fine - it's been acknowledged to the broker, so it's gone from the queue.
The one you've received but not sent on is also good - the broker will re-send it (it will also set the re-delivery flag to give you a hint why it's re-sent it)
The one you've sent on to the downstream app but don't have an ack for is more interesting. How do we know whether it's processed that message? We don't. If we can't can't afford to lose that message then we need to re-send it. That's fine, because we haven't acknowledged it to solace so on re-start the message will be re-sent (with the re-delivery flag set). Won't that mean there's a possibility of a duplicate? Yes. I'm afraid there's no way around that. The broker cannot know the state of this connection. You can get around the duplicate case by putting a sequence id in place for that connection.
I hope that helps. If you want to know more, let us know...
1 -
@TomF & @Aaron
I would like to learn more about exclusive queues and sequence ids. Is there any documentation that gives a bit more detail into how these work - I have read through the Solace docs and I am no more the wiser.
@TomF:
I was thinking about how we would ensure redelivery of the un-acked(it that even a word?) messages. If the message could not be delivered to the downstream application, perhaps due to transient conditions (loss of networks, DNS refresh, server restart), would it be beneficial to implement a back-off and retry pattern, I wonder.
Thanks again for your thoughts - they are certainly very insightful.0
