Ensure only one publisher for a queue?

Harry R
Harry R Member Posts: 3 ✭
in General Discussions #1
If I have a queue that has yet to be populated with messages, and I have multiple instances of a Spring application that have a method to publish messages to this queue upon starting, how can I ensure that only one of these applications will execute the publishing method in order to avoid duplication of messages published to the queue? Is the solution similar to what is given here https://github.com/solacecommunity/spring-solace-leader-election?

Answers

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #2

    Hi @Harry R,

    Looks like this slipped through the cracks. Did you figure it out?

    In general, if there are no upstream events and these apps purely publishers then I think that leader election project is likely the right approach for this scenario. If there are upstream events that trigger the publishing then use an exclusive queue deliver the upstream events so they always go to one instance.

  • Harry R
    Harry R Member Posts: 3 ✭
    #3

    Hi @marc, thanks for the reply.

    Unfortunately not yet. The issue with the leader election option is that the instances have knowledge of each other which makes scaling complicated.

    I have two other ideas, one is to somehow delete all duplicates within a queue however I haven't been able to find any Solace functionality for this → do you know if this is possible (I have only found for preventing receiving duplicated messages)?

    The other is to lock my database row when the instances try to access and send row info to the queue but I have not tried this yet.

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #4

    Hi @Harry R,

    From the Solace broker perspective the messages would be coming from different apps w/ different message IDs, and our brokers don't look at the payload, so there isn't really a way for the broker itself to detect duplicates from a business logic perspective. I think your 3 options are

    1) don't publish duplicates on the provider side…this is the leader election option. Note that by using that spring-solace-leader-election project I do not believe your instances need to know about each other or how many other instances there are. They just have to be able to connect to the broker and make decisions based on whether their flow is active on the exclusive queue.

    2) make sure your consumers can handle duplicates, which you want to do anyway with guaranteed messaging b/c it's guaranteed at least once delivery (not exactly once), or

    3) you have a processor in the middle that receives the messages and republishes to another topic while filtering out duplicates.

    Hope that helps some!

  • Harry R
    Harry R Member Posts: 3 ✭
    #5

    Hi @marc, this has helped a lot, thanks for the advice!

Categories

This Month's Leaders

This Week's Leaders