Hi All,
Is it possible to replicate the messages received on a topic to another topic.
The use case is as follows
producer application puts messages on a topic a/b/c/d with DeliverToOne set.
Whenever a message is put, a copy of that message should go into a topic e/f/g/h ( with DeliverToOne set) on the same vpn
The objective is to split the consuming application into two ( move functionality out of consumer 1 to 2) without producers knowing about it.
Is this possible with some configuration at the vpn ( looking at a low overhead solution)
Have you looked at Shared Subsciptions? What Is Event-Driven Architecture?
Shared Subscriptions is the recommended way of splitting direct messages across consumers and is recommended over DTO as DTO requires the publishers to send a message with that flag set, whereas Shared Subscriptions let’s you handle it completely at the consumer side.
And for your use case: Why do you don’t want to have multiple applications listening to the same topic ? As long as the publisher does NOT set DTO (and I fully agree with @amackenzie and do not recommend setting it), the message is consumed by both applications.
He did say he wants to split processing. So both apps are not supposed to receive the same message. This seems like a perfect situation for shared subs. You’ll need to remove the DTO from your publishers though, I believe.
@vt100, I think we’re a bit confused about what you’re trying to do. It sounds like what you want is 2 deliver to one groups receiving the same messages, so imagine two sets of consumers A and B with 3 consumers each. You want messages to be distributed like this:
Msg 1 goes to A1 and B1
Msg 2 goes to A2 and B2
Msg 3 goes to A3 and B3
Msg 4 goes to A1 and B1
etc.
the consumer application is being split to two consuming applications each doing a specific functionality.
Each consuming application should receive all messages because till the message is parsed it is not known which functionality is to be used ( hence after split a consuming application will ignore a message if it is not meant for its functionality)
When i said application, the application consists of many processes .
The DTO is set so that only one application instance ( 1 process) gets the message and responds. Each application consists of many processes which are listening to the topic. I do not disagree that shared subscriptions can be used instead of DTO ( altough i havent tried it yet)
At the moment there is Consumer1, Consumer2, Consumer3 etc each is a process listening to the topic and responding to the DTO message it gets.
going forward there will be ConsumerA1, ConsumerA2, ConsumerA3 and ConsumerB1, ConsumerB2, ConsumerB3 and so on for the A and B functionality
The publishers cant be changed ( its a big organization)
The reason for using topics is latency , the message has a low time to live, so no point in persistance
Yes right now, there is A1, A2 A3 each a process and one of them should get the DTO message.
going forward one process in A1,A2,A3
and one process in B1, B2, B3 should get the message
How can shared subscription help for this case ?
Lets Assume that A1… A10 and B1…B10 are joining the shared subscription. if a message goes to A1, how can one process in B1…B10 get the same message ?