How to access Solace Cloud statistics and integrate it with monitoring platforms (Datadog)

General Discussions
, ,
1

Hi, I did some research regarding monitoring. Solace offers Insights which is running on Datadog. But it seems it’s a closed system. We want to know how we can access Solace Cloud statistics and send them to public cloud monitoring platforms like Datadog. So we can integrate Solace with existing monitoring solutions of customers and our own monitoring platform we use for Boomi. Is this on the roadmap? We also got this question from several prospects.

2

For collecting broker metrics and send them to a custom monitoring platform, you can use syslog or SEMP to query any of the metrics you are interested in. You can check out more details about syslog forwarding here SEMP and Syslog Monitoring Best Practices which will be very helpful for your usecase.

As of right now, Insights for metrics only go to Insights/datadog so as a short term solution, you can either consider the syslog forwarding and using SEMP as mentioned earlier or use PubSub+ Monitor. Le me know if this is helpful and does what you were looking for

3

Hi @Tamimi ,
Thank you for your reply. SEMP should work I think. But it’s a lot of work to set it up: 1 HTTP calls per metric + parsing. Since Solace Insights is already using Datadog, I wonder if there’s documentation available on how Datadog and Solace have been integrated. For example, with Boomi we setup 1 Datadog .yaml with all the metrics we want to get and Datadog takes care of the rest using JMX. So 1 config and 1 call. That’s it.

4

Hi @sjaak, I see what you mean. We don’t have that kind of yaml file yet.

5

Question: how can we access the syslog of Solace Cloud instances? We want to upload them to Datadog and generate alerts from the logs

6

Aha…I found a hidden feature in the Solace Cloud management console to forward the syslogs to and rsyslog server. You need to click on “Advanced options” on the management page before you see it :smile:

7

@Tamimi We have 1 urgent metric we want to measure, and that the current number of messages in a queue. I checked the SEMP API’s but I can’t find an operation that exposes that information. Customers are chasing us for a solution because this is one of the key KPIs to monitor a message broker in our world. Which is integrating Cloud applications (Finance, CRM, Warehouse Management, HR) using persistent messaging. Any sales order which fails needs to trigger an alert mechanism. Hence Datadog but we need the ability the get to these metrics.

Any ideas are welcome! :smile:

8

@sjaak As far as I can see, what you would need to do is use the monitor SEMP API to get the message meta-data for every message (bear in mind pagination) and programmatically count them.

9

@TomF Looking at the available operations, this is unworkable IMO. I was hoping that there’s an operation for this which simply returns a count with 1 call. Is that on the roadmap?

image.png
image.png2850×992 224 KB

10

Hi @sjaak, success! My colleague @Ramesh_Natarajan_RN has pointed out that you can actually get the message count from the monitor API:
GET http://localhost:8080/SEMP/v2/monitor/msgVpns/default/queues/<Queue Name>
yields

    "collections":{
        "msgs":{
            "count":1
        },
        "priorities":{},
...

You can also query all the queues in your MessageVPN and get the message count for each. Thanks @Ramesh_Natarajan_RN !

11

Nice! Let me have a look. Thanks, @TomF :smile:

12

Thank @Ramesh_Natarajan_RN - he deserves the credit!

13

Thank you @Ramesh_Natarajan_RN ! :smile: Will explore the solution.

14

@Ramesh_Natarajan_RN @TomF This works! This info is crucial for us to manage and sell the Boomi-Solace platform to customers. Thank you! :smiley: