Order of messages in different topics and fairness

General Discussions
1

Hi,
I have a question about order of messages in different topics:

  1. Suppose a service publishes messages A, B into topic T1
    and then messages C, D into topic T2. If a different service
    subscribes (via JCSMP API) to both topics T1 and T2 will it receive messages A, B
    before C, D?

  2. Is there a different answer to 1) if a subscriber uses one subscription
    eg. > instead of two concrete subscription to T1 and T2?

  3. Is there a different answer to 1) if there is some other publisher
    which is at the same time publishing lots of messages into T2?

Thanks

2

Hi @radekm, great question!

Solace offers strong ordering guarantees, but it’s important to remember the limits to those guarantees:

  • Ordering is guaranteed at ingress to the broker. In other words, we can’t guarantee ordering in one broker will be the same in another broker in a mesh, because we can’t guarantee the network between the two broker instances. Also, we can only order things on the basis of their arrival time to the broker - the time they were received, not the time the sender sent them.
  • Ordering is guaranteed across the same topic.
  • If the receiver is a direct subscriber (so a subscriber to a topic, not a consumer from a queue) messages can, of course, be lost to that subscriber in accordance with the DIRECT quality of service, which will disturb ordering to that client.
  • Ordering is guaranteed across fail-overs (HA pair or DR).

Applying these principles to your questions:

  1. In all probability yes, but we can’t guarantee it, and you shouldn’t rely on it. If you’re using the same service to publish to both topics, I can’t think of a case where ordering wouldn’t be preserved, but the problem comes when you have two different publishers, each publishing to a separate topic - there we definitely can’t guarantee ordering across the different topics, which is why you shouldn’t rely on ordering across different topics in the single publisher case.
  2. No, ordering is dependent on the topic to which the messages are published and the order of arrival at the broker. The format of the subscription does not affect the ordering of delivery to a subscriber.
  3. Only if the other publisher affects the order of your first publisher’s messages arrive at the broker. This could happen if the second publisher causes network congestion, for example.
3

Hey @radekm … I have a few different thoughts than Tom, so I’m going to put them here…

I trust that we’re assuming things like: the publisher is publishing all messages from the same thread, the publisher and subscriber both only have one connection (Session) to the broker, all messages are being published with the same quality of service (aka Delivery Mode: Direct/Guaranteed), same Class-Of-Service (1,2,3), and same Priority. And assuming no loss (e.g. due to network drop-out for Direct messaging).

With all those assumptions holding true, then #1) yes a subscriber listening to both t1 and t2 will receive messages in order: A, B, C, D. Because the messages arrived in the broker in that order, because the publisher sent them in that order from a single thread on the same TCP connection.

Ordering is guaranteed _across _topics, in the order that the broker received them. In Tom’s explanation above for #1, he’s talking about two different publishers… and yes, in that case there is/could be a race condition as to which messages arrive at the broker first. But still, once the messages arrive at the broker, all consumers with the same set of subscriptions will receive the messages in the exact same order. (given my assumptions above).

BTW, one other assumption I should have made: if using Guaranteed messaging, the pattern is to have queues for the consumer, and subscribed to topics. The publisher still publishes on Topics, but should ensure the DeliveryMode is Persistetnt. In this case, I’m assuming you only have a single queue subscribed to both t1 and t2. If you had two different queues, then again there could be a race condition as to which messages are received first.

I guess for the #3 question, no there should be no changes to ordering! There might be some change in timing between messages if the broker is being overloaded, and obviously the consumer will be receiving a lot of extra messages on t2, but the absolute order of A, B, C, D will not change.

@TomF am I missing something in what you’re trying to explain?

4

Hi,

@TomF @Aaron , I learned so many concept from above answer .

Thank you so much for detailed answer and well explained concepts . I am sure other people would have got benefited from above answer.

@radekm Good Question Mate

5

@aaron, we’re right on the edge of corner cases here, but assuming t1 and t2 are being sent by different producers, my point was there is a chance of network congestion upsetting the ordering the broker receives the messages from t1 and t2.

6

Steve and I had a chat about this on my Office Hours (link), if anyone is interested…!