Discard indications and topic dispatch
Hi, how does discard indication (in .NET) work together with topic dispatch?
According to the documentation
Message discards can occur if an event broker’s egress per-client priority queues fill up with received messages. When this occurs, the oldest messages on the queues can be discarded to allow new incoming messages to be enqueued. Egress per-client priority queues could fill up in a situation where there are slow subscriber (that is, clients that are not consuming messages quickly enough).
if my egress queue contains
- message A to topic
T(top of the queue) - message B to topic
T2 - message C to topic
T
and gets full then message A will be discarded and discard indication will be set on message B?
But if I have two subscriptions one to single topic T and another to everything >
then the first subscription to T won't get discard indication?
Best Answer
-
Hi @radekm, let's start with some basics. The discard logic is applied to a client. You mention dispatch - topic dispatch is an API level feature that allows you to have separate callbacks for different topics: as far as the broker is concerned, all the topics are going to one client and that's were the logic is applied.
Remember these "queues" are in memory TCP level queues, not anything to do with middleware queues.
And discard indication is set on messages, irrespective of topics. So yes, in the case you mention it's possbile that if message A is discarded, B will arrive with a discard indication. That means the callback for T2 will see the discard, but the callback for T, which will get C, might not see the discard indication. You'll need a way to share the discard indication amongst your callbacks.
Saying that, discard indication is a "hint." You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.
1
Answers
-
Hi @radekm, let's start with some basics. The discard logic is applied to a client. You mention dispatch - topic dispatch is an API level feature that allows you to have separate callbacks for different topics: as far as the broker is concerned, all the topics are going to one client and that's were the logic is applied.
Remember these "queues" are in memory TCP level queues, not anything to do with middleware queues.
And discard indication is set on messages, irrespective of topics. So yes, in the case you mention it's possbile that if message A is discarded, B will arrive with a discard indication. That means the callback for T2 will see the discard, but the callback for T, which will get C, might not see the discard indication. You'll need a way to share the discard indication amongst your callbacks.
Saying that, discard indication is a "hint." You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.
1 -
Ok, makes sense.
You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.
And if I turn off eliding are there other situations when discard indications are not reliable?
BTW how can I mark my question as answered?
0 -
Hey @radekm when you make a post you can actually choose "Ask a question" or "Discussion". Discussion is the default, but if you click that down arrow you can choose "Ask a question". With that option you then get the "Did this answer the question" option on each post. I went ahead and changed this thread to the question type if you'd like to give it a shot.
1
