Partitioned Queues are now Generally Available with 10.4.0

rtomkins
rtomkins Member, Employee Posts: 26 Solace Employee
edited June 2023 in PubSub+ Event Broker #1

Solace Community,


I'm proud to announce that Partitioned Queues are now generally available for all in today's release of 10.4.0!

Partitioned Queues are included in the PubSub+ broker software at no extra charge and are supported by our most recent releases of Solace JCSMP, JMS, CCSMP, Python, Golang, .NET, and JS APIs as well as standard REST/HTTP, MQTT 5.0, and AMQP!

Partitioned Queues are a new subcategory of non-exclusive queues that provide load-balancing and auto-scaling while still providing in-order delivery based on the message context like a retail order, a flight, or a financial transaction! You can think of it as Solace's answer to Kafka Consumer Groups - but better!

Of course, it works well-distributed tracing! You can even see which partition in the partitioned queue the traced message was enqueued for detailed event mesh observability.

For more information please check out my video: https://solace.com/resources/videos/solace-partitioned-queues


Keep it real-time,


Rob Tomkins

Principal Core Product Manager, Solace

P.S. 10.4.0 with partitioned queues is coming to PubSub+ Cloud soon!

Tagged:

Comments

  • Linus
    Linus Member Posts: 8 ✭
    #2

    So we are supposed to use the user property map to set broker relevant metadata?

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #3

    @Linus, that is correct. This allows AMQP and MQTTv5 publishers to also be able to use this feature.

  • rtomkins
    rtomkins Member, Employee Posts: 26 Solace Employee
    edited May 2023 #4

    @Linus, yes. More details on setting the partition key are in our docs page here.

  • jaydensargent
    jaydensargent Member Posts: 2 ✭
    #5

    Is message priority on partitioned queues something that is expected to be supported in the future?

  • rtomkins
    rtomkins Member, Employee Posts: 26 Solace Employee
    #6

    @jaydensargent,

    Typically customers are looking for in-order delivery of messages from a partitioned queue otherwise a non-exclusive queue would be sufficient. Message priority would invalidate in-order delivery.

  • jaydensargent
    jaydensargent Member Posts: 2 ✭
    #7

    @rtomkins Our use case is that we have a dynamic number of partition keys that we want to sequence across, but allow certain messages jump to the front of messages for the same key (effectively resulting in a key not being processed concurrently). Without partitioned queues we would have to manage the mapping of partition key to exclusive queue with a single consumer (resulting in thousands of queues and consumers). Given a partitioned queue maps a single partition to a single flow this achieves everything we need aside from the ability to set priority. Can you elaborate on why it is more typical for customers to want ordered delivery on partitioned queues because our use case doesn't feel atypical?

Categories

This Month's Leaders

This Week's Leaders