Gaps and direct vs guaranteed messaging
Assume I disabled reconnect on Solace session - so once the session is down it will stay down. Now I have a publisher which is publishing messages A, B, C (in order and all messages have the same priority).
(1) Suppose the publisher is publishing to topic T and uses direct messaging.
Can it happen that only messages A and C are delivered to the topic T
and message B is lost?
(2) Suppose the publisher is publishing to queue Q and uses guaranteed messaging.
According to the accepted answer in https://solace.community/discussion/358/pattern-for-handling-rejectedmessageerror-events-with-guaranteed-messages
it could happen that A and C are acked but B is nacked?
If that is the case what are the options for publisher to prevent gaps?
Is it possible to have Publish Windows Size > 1 and still ensure there are no gaps?
Comments
-
Hi @radekm,
Question: Why do you want to disable reconnect on Solace session ?
To your questions.
(1) Suppose the publisher is publishing to topic T and uses direct messaging.
Can it happen that only messages A and C are delivered to the topic T
and message B is lost?If your publisher uses direct messaging and your consumer uses direct messaging (that is: consumer subscribes to a topic, not bind to a queue):
- If your consumer is disconnected during publishing, the messages will be lost
- If your consumer is connected during publishing, it is possible (but unlikeley) that your messages can be lost, e.g. because your consumer is too slow.
It is possible, that your consumer is bound to a queue, that has subscriptions to the topics. In that case, the messages (even if published as direct) are spooled in the queue until the consumer will consume them from the queue. As long as the queue and the subscription exists, spooling will happen, even if the consumer itself is down.
(2) Suppose the publisher is publishing to queue Q and uses guaranteed messaging.
According to the accepted answer in https://solace.community/discussion/358/pattern-for-handling-rejectedmessageerror-events-with-guaranteed-messages
it could happen that A and C are acked but B is nacked?You know, that you can publish to a topic and still use guaranteed messaging ? But the answer to the question is the same.
Yes, it is possible, that A and C are acked and B is nacked.
The most common reasons for nack'ing a message are:
- destination queue is full (if you are publishing to a topic: ... and the (full) destination queue has reject-message-to-sender-on-discard set)
- Publishing ACL violation is set (maybe the topics for A, B,C are NOT identical)
Imaging queue is full after publishing A, and still full during publishing B, but consumer is able to consume some messages before C is published.
Does that answer your question ?
0 -
Thanks for the answer.
Question: Why do you want to disable reconnect on Solace session ?
Because it opens additional opportunity for gaps: Situation when publisher publishes A and B but because of disconnection B is not delivered to the broker. But then publisher reconnects and successfully publishes C.
You know, that you can publish to a topic and still use guaranteed messaging ?
Actually I didn't know it.
Does that answer your question ?
Additionally what can publisher do to prevent gaps which can't be detected by a consumer? Eg. I'm ok if after losing B I lose all messages after it. But I would like to avoid situation that I'll lose some message in the middle or messages are delivered in a different order.
0 -
The typical answer to your requirement is:
- Use guaranteed messaging on both publisher and consumer.
- Publish to a topic, consume from a queue. The queue has the needed subscriptions to spool all messages, where the consumer is interested in.
- Set a high queue quota on the consumer queue to have enough room to store all messages in case the consumer is down
- monitor queue usage. You want to get an alert, if the consumer queue is getting full.
- If you want to ensure, that absolutely no messages are lost between publisher and consumer without NACK to publisher, set "reject-message-to-sender-on-discard" on the consuming queue. With that option, your publisher is receiving a NACK, if e.g. the consumer queue is full or un-available.
- If you need strict message processing on consumer-side in order, you have to set your publish window to 1. If in-order is not a strict requirement, you can increase publish window.
With that setup, you achieve:
- All messages, that are not successfully processed by the broker are NACKed to the publisher
- Even if the consumer is offline, no messages are lost, because they are spooled in the consumer queue
- if your publish window=1, you will receive every NACK immediately and can react on publisher side (most common action: log it and re-try to publish e.g. all 5 seconds until consumer queue is available again).
In addition you have a setup, where a 2nd consumer (eg some statistics engine, that just want to count messages) can consume the same messages without any changes on the publisher.
Uli
2 -
Hi @radekm ,
In addition to Uli's recommendations, I think it's worth highlighting one in particular:
What you really seem to be asking for is some control/detection capability on the publisher side to detect and handle any potential gaps.
You can achieve that by publishing guaranteed messages on a topic and handling the acks/nacks that the broker will respond with to tell your application whether a publish was successful or not.
This is really something all guaranteed publishers should do that care about whether their messages make it to the broker or not. This allows you then to handle a situation where a message may not have been successfully published (for various reasons) and hold back publishing any further messages until that message is successfully retried.Some further reading on the topic I'd recommend:
https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Publishing-Guaranteed-Messages.htmIt's hard to recommend an exact pattern without knowing too much about your use case, but it sounds like you may want to take a look specifically at a blocking guaranteed publisher that sends only one message at a time (in case you need strict ordering).
~~Here's some (Java) sample code you could take a look at to get you started:
https://github.com/SolaceSamples/solace-samples-java-new/blob/main/src/main/java/com/solace/samples/java/sampler/HowToPublishPersistentMessage.java~~Edit: I realise that this example may not be the best one. While I'm looking for the right one, if you basically want to send a message at a time, you'd reduce your publishing window to 1 (so only one message can be send at a time) and handle the acks/nacks accordingly.
Edit2: A nice blog on the topic that might help:
https://solace.com/blog/blocking-vs-non-blocking-publish/Best regards,
Christian1 -
@radekm just adding to Christian and Uli's comments. If you're worried about gaps, as said above, make sure you handle acknowledges correctly.
One option is to set the publish window to one, that is, block publishing until successful receipt of the last message. The penalty is reduced performance.
Another option is to batch up messages in to batches in a transaction. That way you can be sure a given batch of messages have all be successfully received. Again, the disadvantage is a performance penalty.
So, you have a performance vs operational effort trade off to make:
- Option 1: spend the engineering and operational effort to make sure your system functions as designed (e.g. the queue depth is properly monitored to ensure you never encounter a queue full condition; handle disconnections correctly by having the publisher use a Last Value Queue to determine what the last message successfully received was on re-connect). This way you get the performance advantages of windowed guaranteed send;
- Option 2: accept the performance penalty of not being able to use windowed guaranteed send because you need the reassurance that no matter what happens, you will always get messages in order.
3
