Possible to have partitioned queue without rebalancing or handoff?
Partitioned queues are a nice feature. With the hash function, a given consumer always gets the same subset of incoming messages. In my use-case, I never want a consumer to process messages from a different partition. I would prefer for the msgs to fill up their assigned partition and eventually to lose msgs rather than have them sent to a different consumer. So rebalancing and handoff when consumers appear and disappear from a partitioned queue is not needed. Can I turn off these features and always have a given set of messages consumed by a given consumer?
Answers
-
Why use a partitioned queue at all then? Just setup however many queues you need based on number of consumers, configure them to auto-discard if full (not the default), and then use topic subscriptions to shard the incoming stream of data into the appropriate queue (which you can control), vs. relying on the broker's hash function on a key (which you don't control)..?
0 -
Thanks for responding. I find this forum very good for responding.
"Why use a partitioned queue at all then?"
Because I want the hash and mod functions to be applied to a message header which will give me uniform and consistent distribution of messages. Filtering via text matching and wildcards isn't sufficient. I don't think that the hash and mod functions are part of topic subscriptions but perhaps I'm wrong ?
0
