Applying selectors in queue's outbound messages using SolClient.js

tiwaabh

Hi,

I have checked the SolClient documentation link, however I couldn't identify any way of applying selectors while consuming queues messages.

https://docs.solace.com/API-Developer-Online-Ref-Documentation/nodejs/index.html

I undertsand that applying selectors can make the consumption slower. But in our use case, the queue is subscribing to 6 topics. And I want the consumers to read only the specific topic's messages. Rest should remain in the queue.

I have also looked into the discussion, which suggests there is no direct way of filtering messages topic-wise.

https://solace.community/discussion/comment/4309#Comment_4309

So, please suggest if there is a way of applying filter on the outgoing messages from a queue on the basis of topics.

amackenzie

The Solace Javascript API does not support Selectors. You can see that in the matrix table at the bottom of Solace PubSub+ Messaging APIs

Is it possible to add more queues? The magic of pub sub and publishing to topics (instead of queues directly) is that you could add a new queue that only receives 1 of the 6 topics and consume directly from that... same with the other 5. And you can do that without changing your publishers or even the contents of the original 6 subscription queue.

Also, with the PubSub+ architecture, that wouldn't be double the storage consumption as the broker only keeps 1 copy of each message in the spool and it can be referenced by many separate queues.

The only "cost" is 6 queues, 1 for each subscription, which sounds like it ideally suits your use case. Even if JS supported selectors, I think the 6 queue solution is much better for you as there would be no performance penalty.

Aaron

IIRC, you can't perform a Selector operation on the messages' Destination anyhow.

Right, here's a page from our docs: https://docs.solace.com/API/API-Developer-Guide/Using-Selectors.htm pretty much every other message metadata though! I'm guessing this is due to old-school JMS behaviour, where if a message was in a queue, it was b/c it was published directly to that queue, and so every message in that queue would have the same Destination and filtering/selecting based on the message Destination wouldn't make any sense. That's not quite as true for modern Solace practices where a queue can attract multiple topics via subscriptions.

Anyhow, I agree with Andrew, I think 6 queues would be the way to go. 👍🏼

Karl Ossoinig

Another option to consider in the future is a distributed queue. This feature will be release soon, and allows a single queue with messages on one or more topics to be spread across a set of consumers in a "sticky" way based on a key value.

This can be used to load balance processing while ensuring that messages associated with the same key are processed in the order they arrived on the queue. So if you use case has a large or varying number of keys that you are spreading in the topic namespace and attracting to different queues for processing, using a distributed queue instead will make maintaining the solution easier.

tiwaabh

We were avoiding 6 different queue creation as we are creating queues using semp APIs and didn't want to make multiple APi calls in a single process. For 6 queues subscribing to 6 topics would require 12 API calls. Currently, the SEMP API doesn't allow adding multiple queues at once or even adding multiple subscriptions in a queue in a single call.

We wanted to reduce the API calls, if possible. But I think 6 queues is better than 1 queue and then filtering.

Anyways, thanks for the input.

Karl Ossoinig

You're welcome @tiwaabh - That's an interesting design - Sounds like the queues are being dynamically created by the application during runtime, rather than at deployment.

Please reach out to me via LinkedIn if you would like a deeper design consultation, I'm happy to dive into your use case with you to see if there are ways to leverage topics and queues more effectively.

Regards,

Karl

Aaron

If the queues aren't being created in advance and only setup when the apps are deployed, then there's also the option of the client SMF APIs provisioning the queues (e.g. https://github.com/SolaceSamples/solace-samples-java-jcsmp/blob/master/src/main/java/com/solace/samples/jcsmp/features/QueueProvisionAndRequestActiveFlowIndication.java), rather than having to use SEMP.

Tamimi

To add to Aarons answer above, you can create queues and add subscriptions to the queue from the messaging API as well