Multiple Producer and Consumer

Shaz

@marc

Is that possible to publish from multiple Producer and multiple consumer consumes from a single queue ?

amackenzie

I am not @marc (I can only dream to be as smart as that guy afterall) but looking at your question, I think if we broke it down into Producer and Consumer, that might be useful.

Producer

• Multiple producers can publish to the same queue
• There would not be any guarantee of publishing order in this scenario as each publisher would go at its own rate and therefore you could have, for example, Publisher1 sending 3 messages in the time it takes Publisher2 to send 1.

Consumer

• Multiple consumers on a single queue is "possible" but you need to be aware of the characteristics of what they are consuming.
• Exclusive queue:
  • only 1 consumer will be able to bind to the Queue flow and be active at a time. The other consumers can be "connected" but will not be active and therefore will NOT be receiving messages.
• Non-exclusive queue:
  • all connected consumers will receive messages but will each receive different/unique messages in a round-robin from the Queue. E.g. Consumer1 -> msg1, Consumer2 - msg2, Consumer1 -> msg3, Consumer2 -> msg4, ...

The basic rule is that a Queue is meant to persist messages for 1 and only 1 application. If you have multiple applications needing the same messages, you create a queue for each one and each queue subscribes to the same Topic to "attract" the messages to the application queue(s).

This is why publishing to a Topic (instead of a Queue) is so flexible. Have a new application come online that needs the same messages as previous applications? Easy, create a new Queue, add a Topic Subscription to the needed messages and it immediately starts getting messages without ANY impact on your publishers or other applications.

marc

I agree with what @amackenzie said. Maybe he's pretty smart after all 😝

I'll second that you should consider publishing to topics and consuming from queues.

Shaz

@amackenzie @marc

Cool

I just want to publish to a topic and consume the data based on a condition need to publish another two topic. Can you please explain this type of communication with an example.

Thank you

TomF

Hi @Shaz,

I'm not entirely sure what you mean by "publish to a topic and consume the data based on a condition need to publish another two topic." I think what you might mean is publish some data from one publisher, and have two consumers consume different parts of this stream, based on some condition.

If so, the easiest way to do this is to have a field in the topic that contains the data on which your condition needs to match. Let's say you have consumer 1 that looks for messages with condition == true and consumer 2 with condition == false. Have your publisher publish to a topic with a condition field, like this:

some/topic/{true|false} (e.g. some/topic/false)

Consumer 1 can subscribe to some/topic/true (or even * / * /true)

Consumer 2 can subscribe to some/topic/false.

Tamimi

And if what you are referring to is based on a condition after processing your first message you want to publish on another topic, you can follow a processor pattern to choreograph your architecture. So for example:

Publisher A publishes a message on topic1/level1/level2

Processor B subscribes to topic1/level1/level2 and processes the message. Based on a condition, the processor can publish on either topic2/level1/true or topic3/level1/false. This way your publisher subscribes to topics and publishes messages on topics.

(assuming this is what you meant by "publish to a topic and consume the data based on a condition need to publish another two topic.")

Shaz

@Tamimi @TomF

Thank you

The issue is that I want two topic subscription and the consume in a single spring boot app . Is that possible to create multiple consumer for each topic

Tamimi

You can definitely add multiple subscriptions to one app, by simply subscribing to another topic :)

Note that you can leverage Solace Wildcard if you want to subscribe to a range of topics. And if you know that the condition is in the topic hierarchy (like what Tom has mentioned) you can just subscribe to one topic some/topic/* which will attract messages published on some/topic/true and some/topic/false

Shaz

@Tamimi

Thank you 😊

I just want to know how to filter according to topic while consuming from a common consumer started in single spring boot app. Or how to specify or declare different consumer using solace in Java spring boot.

Thank you

TomF

@Shaz, it sounds like you want to configure Sprint Boot to subscribe to two different topics but have messages from each topic processed by different logic - is that right?

@marc, is there something like topic dispatch in Spring?

Shaz

@TomF

Thank you

Please find the attachment for my concept

TomF

Hi @Shaz,

OK, that's clear. There's a few ways of doing this, which I'll put in order of preference.

1. Make the publisher publish to two different topics. This is simplest and offers the best performance
2. Have a "splitter" microservice perform the condition for you and re-publish
3. Add a message header with your condition variable in and use selectors. Please don't do this - it's an anti-pattern.

Shaz

@TomF

Thank you for your valuable response

Could you please explain the point 1 or 2 with an example. I am new to both java and solace

TomF

@shaz,

In one you'll need to edit your publisher to publish to 2 different topics. Taking the HelloWorld sample from github.com/solacesamples, line 97 on, we create two topic strings:

String topicStringTrue = "topic/true";

String topicStringFalse = "topic/false";

if (condition == true) {

publisher.publish(message, Topic.of(topicStringTrue));

} else {

publisher.publish(message, Topic.of(topicStringFalse));

}

This is the best approach so try this one first.