Discard indications and topic dispatch

radekm

Hi, how does discard indication (in .NET) work together with topic dispatch?

According to the documentation

Message discards can occur if an event broker’s egress per-client priority queues fill up with received messages. When this occurs, the oldest messages on the queues can be discarded to allow new incoming messages to be enqueued. Egress per-client priority queues could fill up in a situation where there are slow subscriber (that is, clients that are not consuming messages quickly enough).

if my egress queue contains

• message A to topic T (top of the queue)
• message B to topic T2
• message C to topic T

and gets full then message A will be discarded and discard indication will be set on message B?

But if I have two subscriptions one to single topic T and another to everything >
then the first subscription to T won't get discard indication?

TomF

Hi @radekm, let's start with some basics. The discard logic is applied to a client. You mention dispatch - topic dispatch is an API level feature that allows you to have separate callbacks for different topics: as far as the broker is concerned, all the topics are going to one client and that's were the logic is applied.

Remember these "queues" are in memory TCP level queues, not anything to do with middleware queues.

And discard indication is set on messages, irrespective of topics. So yes, in the case you mention it's possbile that if message A is discarded, B will arrive with a discard indication. That means the callback for T2 will see the discard, but the callback for T, which will get C, might not see the discard indication. You'll need a way to share the discard indication amongst your callbacks.

Saying that, discard indication is a "hint." You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.

TomF

Hi @radekm, let's start with some basics. The discard logic is applied to a client. You mention dispatch - topic dispatch is an API level feature that allows you to have separate callbacks for different topics: as far as the broker is concerned, all the topics are going to one client and that's were the logic is applied.

Remember these "queues" are in memory TCP level queues, not anything to do with middleware queues.

And discard indication is set on messages, irrespective of topics. So yes, in the case you mention it's possbile that if message A is discarded, B will arrive with a discard indication. That means the callback for T2 will see the discard, but the callback for T, which will get C, might not see the discard indication. You'll need a way to share the discard indication amongst your callbacks.

Saying that, discard indication is a "hint." You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.

radekm

Ok, makes sense.

You can't assume that you haven't had discards if you don't see a discard inditcation, and there are cases, such as eliding, where it's possible for a message with a discard notification itself to be discarded.

And if I turn off eliding are there other situations when discard indications are not reliable?

BTW how can I mark my question as answered?

marc

Hey @radekm when you make a post you can actually choose "Ask a question" or "Discussion". Discussion is the default, but if you click that down arrow you can choose "Ask a question". With that option you then get the "Did this answer the question" option on each post. I went ahead and changed this thread to the question type if you'd like to give it a shot.

radekm

@marc mystery solved - thank you!