Trying to understand Consumer Ack Window
I think that I have totally misunderstood the consumer flow ack window, and am trying to figure this out. Does the consumer ack window dictate the max number of messages unacked by the JCSMP API, or by the application?
I have client acknowledgement enabled, and want to throttle the application so that the broker does not allow more than 255 outstanding messages at any time that are unacked by the application. Can I do that via the JCSMP client, or do I have to track that myself?
Comments
-
Hi @petegehrman,
Solace Event Brokers actually allow you to control this in multiple places. Note this can also be referred to as tuning "prefetch"
First, on a queue you can set the MAX_DELIVERED_UNACKNOWLEDGED_MSGS_PER_FLOW, which controls "The maximum number of messages delivered but not acknowledged per flow for the Queue."
The second way is on the flow itself a client can set the FLOW_WINDOW_SIZE which controls how many messages can be in flight from the broker to the API/App at once (1-255).
Note that MAX_DELIVERED_UNACKNOWLEDGED_MSGS_PER_FLOW should always be >= FLOW_WINDOW_SIZE or the FLOW_WINDOW_SIZE won't be taking any affect.
@Aaron @Ragnar , I'm getting dangerous in the details here, can you double check that I didn't say anything in correct and anything else that might be important here 😝
0 -
Thanks. What I'm getting confused about is API acknowledgement vs client acknowledgement. It seems like both of the above window settings relate to API acknowledgement (# of messages in transit on the wire). I want to limit the number of in-flight messages that have not been acked by the client application.
When I receive messages I handle them with a thread pool for parallel processing. However, if I process too slowly, the JCSMP API will continue to receive messages and queue them up in the thread pool's queue. I want to apply back-pressure and prevent the broker/API from delivering more messages at some threshold.
Thanks,
Pete0 -
There is a lot of confusion here. We could do a better job communicating this.
A client acknowledgement merely signal to the broker the message has been received and processed and can now be deleted from the endpoint (topic endpoint or queue). If the message is no longer on any endpoints it is removed from the message spool.
It is not flow control. Withholding acknowledgements will not stop the broker from sending more messages. You can simulate flow control by reducing MAX_UNACKNOWLEDGED_MESSSAGES on the endpoint to a point where the application can stop message flow by withholding acknowledgements. However that is a blunt hammer to solve this. MAX_UNACKNOWLEDGED_MESSAGES is intended to prevent one receiver from consuming too many resources. It is not intended for flow control.
API, or transport, acknowledgements are the flow control mechanism. JCSMP will only allow up to the defined window size (255 by default) + 1 message to be queued on it's receive queue. There will be never more than that many messages in flight at a time as you desire. If you do not call Consumer.receive(), because the application is busy doing other things, then the API will start withholding acknowledgements and reducing the transport window automatically on your behalf.
2
. 
