Trying to understand Consumer Ack Window

petegehrman
petegehrman Member Posts: 43 ✭✭
edited October 2021 in General Discussions #1

I think that I have totally misunderstood the consumer flow ack window, and am trying to figure this out. Does the consumer ack window dictate the max number of messages unacked by the JCSMP API, or by the application?
I have client acknowledgement enabled, and want to throttle the application so that the broker does not allow more than 255 outstanding messages at any time that are unacked by the application. Can I do that via the JCSMP client, or do I have to track that myself?

Comments

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 914 admin
    edited October 2021 #2

    Hi @petegehrman,

    Solace Event Brokers actually allow you to control this in multiple places. Note this can also be referred to as tuning "prefetch"

    First, on a queue you can set the MAX_DELIVERED_UNACKNOWLEDGED_MSGS_PER_FLOW, which controls "The maximum number of messages delivered but not acknowledged per flow for the Queue."

    The second way is on the flow itself a client can set the FLOW_WINDOW_SIZE which controls how many messages can be in flight from the broker to the API/App at once (1-255).

    Note that MAX_DELIVERED_UNACKNOWLEDGED_MSGS_PER_FLOW should always be >= FLOW_WINDOW_SIZE or the FLOW_WINDOW_SIZE won't be taking any affect.

    @Aaron @Ragnar , I'm getting dangerous in the details here, can you double check that I didn't say anything in correct and anything else that might be important here 😝

  • petegehrman
    petegehrman Member Posts: 43 ✭✭

    Thanks. What I'm getting confused about is API acknowledgement vs client acknowledgement. It seems like both of the above window settings relate to API acknowledgement (# of messages in transit on the wire). I want to limit the number of in-flight messages that have not been acked by the client application.

    When I receive messages I handle them with a thread pool for parallel processing. However, if I process too slowly, the JCSMP API will continue to receive messages and queue them up in the thread pool's queue. I want to apply back-pressure and prevent the broker/API from delivering more messages at some threshold.

    Thanks,
    Pete

  • Ragnar
    Ragnar Member, Employee Posts: 64 Solace Employee

    There is a lot of confusion here. We could do a better job communicating this.

    A client acknowledgement merely signal to the broker the message has been received and processed and can now be deleted from the endpoint (topic endpoint or queue). If the message is no longer on any endpoints it is removed from the message spool.

    It is not flow control. Withholding acknowledgements will not stop the broker from sending more messages. You can simulate flow control by reducing MAX_UNACKNOWLEDGED_MESSSAGES on the endpoint to a point where the application can stop message flow by withholding acknowledgements. However that is a blunt hammer to solve this. MAX_UNACKNOWLEDGED_MESSAGES is intended to prevent one receiver from consuming too many resources. It is not intended for flow control.

    API, or transport, acknowledgements are the flow control mechanism. JCSMP will only allow up to the defined window size (255 by default) + 1 message to be queued on it's receive queue. There will be never more than that many messages in flight at a time as you desire. If you do not call Consumer.receive(), because the application is busy doing other things, then the API will start withholding acknowledgements and reducing the transport window automatically on your behalf.

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 914 admin

    Thank you @Ragnar. I agree we can probably do a better job communicating this :p.
    Now we at least have a community post that shares it!