application going on non responsive mode after Consumers window size 0

application going non responsive mode after Consumers window size 0 under the Queue configuration
I am not sure why window size is getting 0 .
could you please help anyone , when the window size will be 0 and how to resume application after window size 0 ?

Comments

  • malayarasanm
    malayarasanm Member Posts: 2

    please refer the below screenshot for reference

  • TomF
    TomF Member, Employee Posts: 406 Solace Employee

    Hi @malayarasanm, this window size 0 condition generally means your application as a whole is unresponsive. The thread that processes incoming messages and sends acknowledgements (the context thread) is blocked. The Solace API can't read any more messages from the network and isn't sending any acknowledgements back.
    Do you have the output from any profiling tools for your application, such as top? Is that showing your CPU is at 100%? Is your application producing any output? Is it stuck waiting for something else to happen, like obtain a lock to a database? Do you have any heart beat mechanism in your application? I can see that it has received 195 messages successfully.

  • Jordi
    Jordi Unconfirmed, Member Posts: 11

    We are having the same problem.

    We see neither a flow event nor a session event being triggered. CPU is fine and it seems to affect sessions randomly.

    What helps is restarting the session, but it is not satsifying.

    We use the SolaceSystems.Solclient.Messaging version 10.19.0 for our application.

  • TomF
    TomF Member, Employee Posts: 406 Solace Employee

    @Jordi There won't be any flow or session event. This is the Solace API's flow control mechanism, when the broker senses that the application is busy and stops sending messages until the application is ready to receive again.

    There are 2 layers of acknowledgement in a flow, the transport ack (message had made it to the API and has been presented to the application) and application acknowledgement (application has finished processing the message and it should be deleted).

    What's happening is that the Solace API isn't sending transport level acks back to the broker. Either the network connection is closed or the API is blocked.

    In @malayarasanm's screenshot above, there are 19 messages outstanding that haven't been acknowledged by the application. Do you have this output in your case? Even more helpful, if you have access to the CLI, run the

    show client <client name> connections wide

    command.

  • Jordi
    Jordi Unconfirmed, Member Posts: 11

    Wow. Did not expect to receive such a quick response to the 3 year old thread. Nice!

    Yes, we see pretty much the same as in  @malayarasanm's screenshot above. The unackkowleded messages match our MaxUnackedMessages threshold.

    Why we cannot acknowldege messages is still unclear, but we try to detect, when WindowSize become zero to restart the flow.

    Any idea how we could detect WindowSize becoming 0 or get the number of unackkowleded messages?

    The flow.Properties.WindowSize property does not work as it sets only the max WindowSize.

    Which WindowSize is displayed in the Web interface above?

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 508 admin
    edited April 2023 #7

    Hey @Jordi. If you're using the C# API, then I'm wondering if your app is doing "work" in the message received callback? One issue people run into with our C and C# APIs is that the thread that calls the event handler for your IFlow is owned by the API, and is the only thread the API has for doing its other work. So if the client app blocks up this thread for a long time (e.g. trying to write to disk or database or some other service), the functionality of the API can suffer, and which is why you might be seeing a window size of 0 on the broker. To help fix this, your callback event handler should take the message and place it in some linked list or ring buffer or something and return quickly, and another application owned/created thread should process the messages from there and ack() them when done. This leave the API context thread to continue working.

    If, however, your queue is showing that unacked messages is equal to your max unacked setting on the queue, that means that the queue is blocked sending your consumer because your app is not ACKing messages. It needs to do that when it is done processing/storing them.

  • Jordi
    Jordi Unconfirmed, Member Posts: 11

    Hi Aaraon,

    we process messages exactly as ypu propose: the  message event handler puts the message in a list (a ConcurrentBag actually) and another Task/Thread is processing the list and acknowleding the messages.