Load Balancing on One Topic Multiple Consumer
Hi all, I am new to Solace.
Recently I am working on a project that assume the publisher only work on one topic and there is multiple consumer listen to the same topic.
The project will be deployed to a k8s and I expect there would be a scaling on the consumer side.
My design at this point is one publisher publishing a topic to multiple queue so the message will spread through the queues. However I would like to have unique message in each queues which means if MessageA is assign to Queue1, MessageA won't appear in another queue.
I would like to know the numbers of outstanding message in order to formulate my scaling rules for k8s.
For this architecture, how should I configure in order to achieve the goal? For further information, I am using python to compile the program.
I would be grateful if someone can provide me with suggestions or guidance. Thank you very much.
Best Answer
-
Hi @BrianChan,
the typical Solace solution for this is ONE queue with multiple consumers listening.
Each message in this queue is forwarded round-robin to one of the listening consumers.
You can add and remove consumers any time you want.
Just set the queue to "non-exclusive".
Feel free to ask, if anything is unclear here.
1
Answers
-
Hi @BrianChan,
the typical Solace solution for this is ONE queue with multiple consumers listening.
Each message in this queue is forwarded round-robin to one of the listening consumers.
You can add and remove consumers any time you want.
Just set the queue to "non-exclusive".
Feel free to ask, if anything is unclear here.
1 -
Hi @BrianChan
Just to add for scaling, you can leverage KEDA (https://keda.sh/docs/2.6/scalers/solace-pub-sub/) for Solace, which monitors either message count or spool size.
1
