Consume a persistent Message from Queue or Subscription

murillocg

After some studies, and proof of concept, I have a question about the consuming side, since the producer side is very clear to me.

Let's say I need to save files using Solace Message Broker, so I have the following configuration for producer, topic, and queue:

The producer sends a message with the Topic "storage/document/save-document". This topic is assigned to Queue "storage.document.save-document.queue" via Solace REST API. So, when a message is sent to this topic, it stays in this queue until a consumer consumes it.

Question 1)

Here, you say the best practice is consuming from a Queue, is that always so? In this case, what about the flexibility to plug an analytics system here to collect statistics? How I should build this?

Question 2)

I can't use the wildcards to consume persistent messages, for instance, create a consumer to log all storage events matching a range of topics: "storage/>"? How I should build this?

Question 3)

Is the subscription available in the Solace REST API between a topic and a queue, the same called here? `jcsmpSession.addSubscription(queue, topic, JCSMPSession.WAIT_FOR_CONFIRM)?`

uherbst

Hi @murillocg

A1): The recommendation is still "publish to topic, consume from a queue". There are more options than that available, but you should have good reasons to do it in another way.

Please describe your use case with analytics system. What statistics do you want to collect ?

If its mainly "How many messages with this topic were published", you don't need to consume the messages at all. It should be good enough to query the queue statistics via SEMPv2.

A2. For sure, you can use wildcards for queue subscriptions.

No reason not to add a subscription "storage/>" to a queue.

What have you tried, that did not work ?

A3: We have to be careful about the term "Solace REST API". Solace has more than one...

There is one API to publish messages via REST and there is one API to change things in the broker (like adding a new subscription to a queue) or to query metrics. We call this API "SEMPv2".

I assume, you talk in Q3 about SEMPv2. The subscriptions made programmatically in your jcsmp-queue result in exactly the same subscriptions as a SEMPv2-call.

murillocg

Hello @uherbst

Q1) The analytics system is something related to my documents, extracting content and storing it in a database, etc. In this case, I still need persistent messages. How I should build this?

Q2) I though I needed just to configure my new consumer to listen to this subscription "storage/>", instead of adding this subscription to the queue on the broker side.

Q3) So, I'm talking about the API "SEMPv2". Ok. Thanks.

uherbst

A1): Do I understand you correctly: You have two consumers ? One is your analytics system, one is a consumer, that just store that payload into a file somewhere ?

In that case: Use two apps with 2 queues with the same subscription. With that construct, each message is stored in each queue and can be consumed independently.

A2) Depends:

You need a queue with subscription(s) (can be multiple subscriptions in one queue).

You can either do that administratively (with SEMPv2) or you can do it as part of your messaging API (as you have done with JCSMP). Both is valid. There is no recommendation to use one over the other (for many enterprises, there are company regulations about that).

BTW: If you use protocols like AMQP, there is no way to create queues and subscriptions as part of the messaging API. In that case, you have to use the SEMPv2-way.

Tamimi

And to add to @uherbst's answer, it is recommended to use the Management API (SEMPv2) for management/admin related tasks (for example creating queues, configuring your broker...etc) and Messaging APIs for messaging related functionalities (e.g. publishing, subscribing, handling messaging errors, acknowledging messages...etc). For the messaging APIs you can use any of Solace's APIs or an open standard messaging protocol.

We show that in our tutorials as well https://tutorials.solace.dev/

Note: for your comment on consuming from queues, your subscriber will "bind" to a queue and consume the messages from there. The queue will "subscribe" to the topic of choice and attract messages with matching topic subscription.