Horizontal scaling of event brokers?

petegehrman
petegehrman Member Posts: 43 ✭✭

I am investigating creating a HA PubSub+ broker setup, but a single primary broker will not have enough capacity for my needs. Do I need to create multiple triplet clusters and connect them? If so, how is load distributed between the clusters?

Tagged:

Comments

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 916 admin

    Hi @petegehrman,
    You're definitely on the right track there. The default answer here would be to standup multiple brokers and setup an Event Mesh which will leverage DMR (Dynamic Message Routing) to route messages between brokers only when a subscriber wants them. A bit more info in docs here. If you want to discuss your use case in more detail to make sure you're on the right path just let me know and I can connect you with someone that has the conversations with users all the time :)
    -Marc

  • petegehrman
    petegehrman Member Posts: 43 ✭✭

    Thanks @marc. If I have multiple brokers connected via DMR, can I abstract that away from clients and provide a single address, or do publishers/consumers need to explicitly connect to one of the brokers?

  • TomF
    TomF Member, Employee Posts: 406 Solace Employee

    Hi @petegehrman, it all depends on the message quality of service (DIRECT vs PERSISTENT). For DIRECT messages a client can connect to any point in the mesh: there is complete abstraction.

    For PERSISTENT messages, a client needs to connect to a queue. It's therefore difficult to completely automatically abstract persistent message consumption from location in the mesh. With a bit of administrative cleverness, though, this can be done. We're working on a feature to completely automate this.

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 916 admin

    Thanks @TomF. Hope that helps @petegehrman!

  • petegehrman
    petegehrman Member Posts: 43 ✭✭

    Yes, thanks both of you for the help.