Possible to have partitioned queue without rebalancing or handoff?

TomC

Partitioned queues are a nice feature. With the hash function, a given consumer always gets the same subset of incoming messages. In my use-case, I never want a consumer to process messages from a different partition. I would prefer for the msgs to fill up their assigned partition and eventually to lose msgs rather than have them sent to a different consumer. So rebalancing and handoff when consumers appear and disappear from a partitioned queue is not needed. Can I turn off these features and always have a given set of messages consumed by a given consumer?

Aaron

Why use a partitioned queue at all then? Just setup however many queues you need based on number of consumers, configure them to auto-discard if full (not the default), and then use topic subscriptions to shard the incoming stream of data into the appropriate queue (which you can control), vs. relying on the broker's hash function on a key (which you don't control)..?

TomC

Thanks for responding. I find this forum very good for responding.

"Why use a partitioned queue at all then?"

Because I want the hash and mod functions to be applied to a message header which will give me uniform and consistent distribution of messages. Filtering via text matching and wildcards isn't sufficient. I don't think that the hash and mod functions are part of topic subscriptions but perhaps I'm wrong ?

Tamimi

Hey @TomC, do you have any control on the topic the events are being published on? You can add different topic subscriptions to queues and as Aaron mentioned every queue will only attract events that match subscription of choice.

Note that the hashing function with partition queues works on they key of the message found in the header such that all messaged with the same key will be allocated to the same partition, but as you mentioned, that does not mean the partition will only have messages of the same key.

The uniform consistent distribution of messages you refer to can also be achieved with topic subscriptions on separate queues