Prevent Queue subscribing application to receive another message until ACK is sent

brigadier90

Hi there,

I have a consuming application using the solace .NET client API which binds to an exclusive queue.

May aim is to process messages in order without any loss.

Suppose I have the following messages on the Queue: msg1, msg2, and msg3. Let's say I have received msg1 and am processing it on the FlowMessageHandler in my application. However, the processing time may be really long due to the fact that it is dependent on many other external services which DB connections, REST APIs etc. it is also possible that one of these services may not be available.

Questions:

Is there a time limit on how long I can keep retrying to connect to these services in the FlowMessageHandler?

If there is or it is not recommended to hang in the handler for too long, then if I exit the handler without ACKing msg1 is there any way to prevent sending msg2 until msg1 is ACKed?

Can it be done via the max-delivered-unacked-msgs-per-flow option by setting it to 1? if so can this be set via the .NET client API?

If I exit the handler without ACKing msg1 what will happen if the queue receives a msg4? will I get sent msg1 again?

Any help would be greatly appreciated.

Thank you very much

TomF

Hi @brigadier90, and a quick complement on your name choice

Is there a time limit on how long I can keep retrying to connect to these services in the FlowMessageHandler?

TL;DR: yes, return from the FlowMessageHandler delegate ASAP. There's no (enforced) time limit on how long you can have a message waiting for an ack.

The strict answer to this is "no," there is no limit to the time you can pause in FlowMessageHandler. The real answer is yes, you should spend the absolute minimum time in the handler. This is because the handler runs in the context thread, which deals with all the communication via the messaging socket. If you block this thread, no communication can happen the socket, and that's not a good place to be. The best advice would be to place the message in a datastructure, such as an in memory queue, return, and have an application thread do the processing.

If there is or it is not recommended to hang in the handler for too long, then if I exit the handler without ACKing msg1 is there any way to prevent sending msg2 until msg1 is ACKed?

Yes. In your flow properties, set the AckMode to ClientAck and set MaxUnackedMessages to 1. This effectively creates a blocking scenario, where the broker will wait for you to acknowledge a message before sending the next. Very handy for ordering guarantees, but beware, not so handy for performance, which will be degraded considerably.

if so can this be set via the .NET client API?

Yes, MaxUnackedMessages is a flow property. Set it when you create the flow.

If I exit the handler without ACKing msg1 what will happen if the queue receives a msg4? will I get sent msg1 again?

No. Exting the handler doesn't do anything to the message flow unless you have AckMode set to Auto, which automatically acknowledges messages and makes me upset (because you shouldn't do it). What you should do is set AckMode to client, quickly pass the message you picked up in the handler to the application or some other thread, and exit the handler. As this point you have the message but it hasn't been acknowledged. Do what ever processing needs to be done on the message. Take your time (both in writing the code and in execution time) to make sure you really have finished with the message. When you're sure you've done everything you need to do, manually call ack.

An unacknowledged message will only be resent if the flow is destroyed and another binds to the queue.

brigadier90

Hi @TomF,

Thank you very much for your response

I like the approach of acknowledgements from another worker thread! Will attempt to implement it like that.

Best regards

TomF

Glad to have helped @brigadier90. Let us know how you get on!

TomF

Hi @brigadier90,

A follow up thought. You are now putting messages in an in-memory queue between your flow delegate and worker threads. This means you DON'T need to set your flow ack window to 1.

What you should do instead is set the flow window to be the same as the number of messages you can hold in your in-memory queue. That way you use the performance of windowed messages, but flow control via acks. You only process one message at a time from your in-memory queue, so avoid any ordering issues.