Messages getting discarded from broker
Hi ,
We have a jscmp producer publishing messages to a broker. the publisher pushes around few 100 messages and we see messages gets discarded every now and then. we did some debugging and found that "Message Spool Ingress Discards" is getting increased.
Further in show message-spool stats details Publisher Not Found . Could you let know what could be the reason of messages getting discarded and how to overcome it .
Producer : jscmp
Pattern : Topic to Queue
Followed the producer example from here --> https://github.com/SolaceSamples/solace-samples-java/blob/master/src/main/java/com/solace/samples/TopicPublisher.java
Thanks
Rajesh
Comments
-
Hello @kdrajeshbabu , Interesting ... Is is likely the publisher is exiting too soon before the acks are processed ? Can you add some sleep before exiting from the publisher ?
0 -
Hi Rajesh. That's an odd stat to be incremented. If you're just using the sample "as is" it is publishing the messages as Direct (instead of Guaranteed) and the messages are getting promoted onto the queue. Not sure why it would be complaining about the publisher not being found. Can you please share your (modified) source code?
Perhaps take a look at the following sample: https://github.com/aaron-613/solace-samples-jcsmp/blob/master/src/main/java/com/solace/samples/patterns/GuaranteedPublisher.java
0 -
Thanks @nram @Aaron , we were able to get the messages in the broker by adding a sleep. But i have not yet got time to check if the messages were ack'ed or not. Will debug further to check why client was closing soon but felt this weird as dont get any error on the producer.
We were using a Guaranteed delivery mode and the similar to the one you have shared here above(except to the sleep timer). Also in our scenario we get source messages one at a time , each execution delivering one message at a time from source to solace.0 -
Hello @kdrajeshbabu , to your question "why dont get any error on the producer", do you have reject-message-to-sender-on-discard property enabled on the queue ?
https://docs.solace.com/Configuring-and-Managing/Configuring-Queues.htm#Message-Discard-Handling0 -
Hello @nram , yes we enabled the property reject-message-to-sender-on-discard. But still no errors and messages were discarded with no errors on the producer client.
0 -
Hey @kdrajeshbabu . Ok, so yeah, the reason you were getting messages discarded by the broker (and the particular error message you were seeing) is simply because your publishing application was quitting too quickly. By the time the broker had received your persistent message and was ready to spool them, the publishing application had already quit.
Recall: the point of Guarantee of delivery is when the publishing application receives the ACK from the broker. Check this: https://www.youtube.com/watch?v=5wXv2QqVK3U
And therefore the reason you were not seeing any error messages or anything was because the application had already disconnected.
One useful Solace feature for determining which message was last received by Solace (in the event your publisher quits/crashes unexpectedly) is the Last-Value Queue. Check: https://docs.solace.com/PubSub-Basics/Endpoints.htm#LVQs
Hope that helps!
0 -
BTW, we were talking about this issue on my Office Hours livestream (link), and I thought we had figured it out, but from what our internal Support people told us afterwards, the discards were due to a sanity check before spooling the messages, and the publishing application had already disconnected. As mentioned above, please confirm successful delivery by waiting for the ACK (aka
responseReceivedEx()in JCSMP) before disconnecting from the broker.1 -
Great @Aaron .. liked the way you guys discussed on this issue. Currently we have incorporated with a sleep and then wait for the ack .
1
