🎄 Happy Holidays! 🥳
Most of Solace is closed December 24–January 1 so our employees can spend time with their families. We will re-open Thursday, January 2, 2024. Please expect slower response times during this period and open a support ticket for anything needing immediate assistance.
Happy Holidays!
Please note: most of Solace is closed December 25–January 2, and will re-open Tuesday, January 3, 2023.
Solace Topic Destination Header
Hi Team - this is regarding the scenario of publisher using topic string as destination header (using topic hierarchy) and consumer using wild card (> or *) to consume required message. This is in order to facilitate filtering at event broker level.
Does the above scenario ensure guarantee message delivery when none of the consumer is online?
Example: Publisher sends below messages (there is no end point created in broker)
Msg1 Destination Header: OMS/EQ/AK/1
Msg2 Destination Header: OMS/EQ/AK/2
Msg3 Destination Header: OMS/EQ/AS
Consumer1 has 2 instances subscribing to 'OMS/EQ/>'. Consumer2 has 2 instances subscribing to 'OMS/EQ/AK/>'. Assume that both instances of Consumer2 are down.
Consumer1 receives all 3 messages. But since Consumer2 has both instances down, does Solace event broker store those 2 messages (with header OMS/EQ/AK/1 and OMS/EQ/AK/2) and deliver them to Consumer2 once it comes online?
Comments
-
Hi there @sdeveloper. Thanks for the detailed question! So, first thing: use of topics for message filtering == great, but this is only for addressing... if you want Guaranteed delivery / persistent messaging, then you'll also need to be using Queues to store the messages in the broker. That is: your messages should always be published to topics, but if you want Guaranteed delivery, then the publisher will have to set DeliveryMode to
PERSISTENT(or use a PersistentMessagePublisher if using our new APIs), and the consumers will need to use Queues that are subscribed to the topic(s) that they want. Which API are you using, so I can point you to some relevant samples? If your consumer apps subscribe directly to the topics, then you aren't using Guaranteed delivery.In your example, you have two consumers apps, with two instances each? Are these expected to be round-robin, or active-standby?
So if your Consumer1 apps had a queue (
q1) subscribed toOMS/EQ/>, and Consumer2 apps had another queueq2subscribed toOMS/EQ/AK/>and they were both down, then the broker would hold any messages for Consumer2 apps inq2until at least one of them connected to the broker to receive it. Each queue would receive a copy of the message, and if Consumer1 apps were online they'd receive it right away fromq1.Hope that helps! Let us know if you have more questions.
1 -
Thanks a lot @Aaron for your insightful feedback. In order for this to work, it is my understanding that I need to do following. Please let me know if this makes sense.
1. Create Topic Endpoints: 'OMS/EQ/AK' and 'OMS/EQ/AS'
2. Create Queue: 'OMS/EQ' with mapped topics 'OMS/EQ/AK' and 'OMS/EQ/AS'
3. The publisher will send following messages:
a. Msg1 Destination Header: OMS/EQ/AK (I think I can no longer send OMS/EQ/AK/1)
b. Msg2 Destination Header: OMS/EQ/AK (I think I can no longer send OMS/EQ/AK/2)
c. Msg3 Destination Header: OMS/EQ/AS
4. Consumer1 having 2 instances subscribes to queue 'OMS/EQ' and uses wild card of 'OMS/EQ/>' to get all 3 messages
5. Consumer2 having 2 instances subscribes to queue 'OMS/EQ' and uses wild card of 'OMS/EQ/AK' to get only first 2 messages. I think I cannot make this more granular.
I am using Spring Cloud Stream 3.0.7 and Stream Listener based annotation for consumer (not yet on functional approach). Please do share any code sample if you have, especially the configuration for filtering.
0 -
One big thing to remember here is that you do not configure Topics in Solace... topics are ephemeral metadata on each published message. Watch this: https://www.youtube.com/watch?v=PP1nNlgERQI
Topic Endpoints and Queues have quite similar functions, they are both persistent endpoints to store messages. Typically, Topic Endpoints are only used in JMS applications as an analog for durable subscriptions. We recommend Queues with topic subscriptions for most use-cases. See this blog by Leah: https://solace.com/blog/queues-vs-topic-endpoints/
So skip step #1, and everything else should work fine. 👍🏼
The Spring Cloud Stream binder helps create queues and adds topic subscriptions to them automatically. I believe there is a way to tell it to use premade/predefined queues. I'm not too familiar with it. Check the docs here: https://github.com/SolaceProducts/solace-spring-cloud/tree/master/solace-spring-cloud-starters/solace-spring-cloud-stream-starter
1 -
Thanks again @Aaron! I will try this out and update.
0 -
And to add to what @Aaron mentioned, a couple of notes on terminologies:
- You create queues on solace and add topic subscriptions to the queue. This basically means that any msg published to the broker that matches the topic subscription will be attracted to the queue --> Guaranteed messaging
- You can still publishes messages on
OMS/EQ/AK/1from your publisher and addingOMS/EQ/AK/*orOMS/EQ/AK/>as topic subscriptions to your queue - Consumers dont really "subscribe" to queues they bind to a queue and consume messages from it
1 -
Thanks @Tamimi for the clarification!
1 -
@TamimiI tried to test above setup. I think it worked on publisher side but running into issues on consumer side. Please find below details. Can you please kindly review and let me know your thoughts?
1. The NBS.POC.EQ is a pre-created queue with pre-created topics, OMS/EQ/AK and OMS/EQ/AS
2. Publishing side properties (1_Publisher_Properties.JPG) and code (2_Publishing_Code.JPG) is attached. I tested it with Solace 'Try Me' and it worked. Attached is the screenshot using the subscriber of 'Try Me', 3_Subscriber_TryMe.JPG. It shows the expected topic for different message (messages sent from REST end point to sendMessage).
3. I then tried to programatically consume the same set of messages. I tried with multiple version of spring-cloud-starter-stream-solace. Same consumer properties were set across different versions. Attached is the 4_Consumer_Code.JPG
a. spring-cloud-starter-stream-solace version of 2.0.0-string-serialization
Attached is 5_Consumer_Properties_Solace2.0.0.JPG. The consumer gets all messages and the property 'queueAdditionalSubscriptions: OMS/EQ/AS' is not applied.
b. spring-cloud-starter-stream-solace version of 3.2.0
Attached is 6_Consumer_Properties_Solace3.2.0.JPG. For this, I get error on start up of consumer application, 403: Permission Not Allowed. Attached is the stack trace, 7_Consumer_Error_Solace3.2.0.txt.
c. spring-cloud-starter-stream-solace version of 3.3.0
Same properties, as listed in 6_Consumer_Properties_Solace3.2.0.JPG was used. For this, I get error on start up of consumer application, EL1008E: Property or field 'isAnonymous' cannot be found on object of type 'com.solace.spring.cloud.stream.binder.provisioning.ExpressionContextRoot' - maybe not public or not valid?. Attached is the stack trace, 8_Consumer_Error_Solace3.3.0.txt.
Thanks!
0
