Solace Community is getting a facelift!

On March 3rd we will be starting the process of migrating Solace Community to a new platform. As a result, Solace Community will go in to a temporary read-only state. You will still be able to come onto Solace Community and search through posts to find answers, but you won't be able to ask questions, post comments, or react in any way.

We hope to have the migration complete by Wednesday March 5th (or sooner), so please keep an eye out!

Load Balancing on One Topic Multiple Consumer

BrianChan
BrianChan Member Posts: 18 ✭
edited April 2022 in PubSub+ Event Broker #1

Hi all, I am new to Solace.

Recently I am working on a project that assume the publisher only work on one topic and there is multiple consumer listen to the same topic.

The project will be deployed to a k8s and I expect there would be a scaling on the consumer side.

My design at this point is one publisher publishing a topic to multiple queue so the message will spread through the queues. However I would like to have unique message in each queues which means if MessageA is assign to Queue1, MessageA won't appear in another queue.

I would like to know the numbers of outstanding message in order to formulate my scaling rules for k8s.

For this architecture, how should I configure in order to achieve the goal? For further information, I am using python to compile the program.

I would be grateful if someone can provide me with suggestions or guidance. Thank you very much.

Best Answer

  • uherbst
    uherbst Member, Employee Posts: 131 Solace Employee
    #2 Answer ✓

    Hi @BrianChan,

    the typical Solace solution for this is ONE queue with multiple consumers listening.

    Each message in this queue is forwarded round-robin to one of the listening consumers.

    You can add and remove consumers any time you want.

    Just set the queue to "non-exclusive".

    Feel free to ask, if anything is unclear here.

Answers

  • uherbst
    uherbst Member, Employee Posts: 131 Solace Employee
    #3 Answer ✓

    Hi @BrianChan,

    the typical Solace solution for this is ONE queue with multiple consumers listening.

    Each message in this queue is forwarded round-robin to one of the listening consumers.

    You can add and remove consumers any time you want.

    Just set the queue to "non-exclusive".

    Feel free to ask, if anything is unclear here.

  • BrianChan
    BrianChan Member Posts: 18 ✭
    #4

    @uherbst

    Thank you so much for your reply. Actually I have tried non-exclusive queue but in my first trial, it seems all messages go to all consumer. However, I tried again and the message seems forward to either one consumer only. My bad.

    Thanks again.

  • arih
    arih Member, Employee Posts: 125 Solace Employee
    #5

    Hi @BrianChan

    Just to add for scaling, you can leverage KEDA (https://keda.sh/docs/2.6/scalers/solace-pub-sub/) for Solace, which monitors either message count or spool size.

  • BrianChan
    BrianChan Member Posts: 18 ✭
    #6

    @arih

    Thank you for your suggestion. It helps a lot!!

Categories

This Month's Leaders

This Week's Leaders