Try PubSub+

How to access Solace Cloud statistics and integrate it with monitoring platforms (Datadog)

sjaaksjaak Member Posts: 74 ✭✭✭

Hi, I did some research regarding monitoring. Solace offers Insights which is running on Datadog. But it seems it's a closed system. We want to know how we can access Solace Cloud statistics and send them to public cloud monitoring platforms like Datadog. So we can integrate Solace with existing monitoring solutions of customers and our own monitoring platform we use for Boomi. Is this on the roadmap? We also got this question from several prospects.

Comments

  • TamimiTamimi Member, Administrator, Employee Posts: 197 admin

    For collecting broker metrics and send them to a custom monitoring platform, you can use syslog or SEMP to query any of the metrics you are interested in. You can check out more details about syslog forwarding here https://docs.solace.com/Best-Practices/Monitoring-BP.htm which will be very helpful for your usecase.

    As of right now, Insights for metrics only go to Insights/datadog so as a short term solution, you can either consider the syslog forwarding and using SEMP as mentioned earlier or use PubSub+ Monitor. Le me know if this is helpful and does what you were looking for

  • sjaaksjaak Member Posts: 74 ✭✭✭
    edited January 4 #3

    Hi @Tamimi ,
    Thank you for your reply. SEMP should work I think. But it's a lot of work to set it up: 1 HTTP calls per metric + parsing. Since Solace Insights is already using Datadog, I wonder if there's documentation available on how Datadog and Solace have been integrated. For example, with Boomi we setup 1 Datadog .yaml with all the metrics we want to get and Datadog takes care of the rest using JMX. So 1 config and 1 call. That's it.

  • TomFTomF Member, Employee Posts: 246 Solace Employee

    Hi @sjaak, I see what you mean. We don't have that kind of yaml file yet.

  • sjaaksjaak Member Posts: 74 ✭✭✭

    Question: how can we access the syslog of Solace Cloud instances? We want to upload them to Datadog and generate alerts from the logs

  • sjaaksjaak Member Posts: 74 ✭✭✭
    edited February 18 #6

    Aha...I found a hidden feature in the Solace Cloud management console to forward the syslogs to and rsyslog server. You need to click on "Advanced options" on the management page before you see it :smile:

  • sjaaksjaak Member Posts: 74 ✭✭✭
    edited March 1 #7

    @Tamimi We have 1 urgent metric we want to measure, and that the current number of messages in a queue. I checked the SEMP API's but I can't find an operation that exposes that information. Customers are chasing us for a solution because this is one of the key KPIs to monitor a message broker in our world. Which is integrating Cloud applications (Finance, CRM, Warehouse Management, HR) using persistent messaging. Any sales order which fails needs to trigger an alert mechanism. Hence Datadog but we need the ability the get to these metrics.

    Any ideas are welcome! :smile:

  • TomFTomF Member, Employee Posts: 246 Solace Employee

    @sjaak As far as I can see, what you would need to do is use the monitor SEMP API to get the message meta-data for every message (bear in mind pagination) and programmatically count them.

  • sjaaksjaak Member Posts: 74 ✭✭✭

    @TomF Looking at the available operations, this is unworkable IMO. I was hoping that there's an operation for this which simply returns a count with 1 call. Is that on the roadmap?

  • TomFTomF Member, Employee Posts: 246 Solace Employee

    Hi @sjaak, success! My colleague @Ramesh Natarajan -RN has pointed out that you can actually get the message count from the monitor API:
    GET http://localhost:8080/SEMP/v2/monitor/msgVpns/default/queues/<Queue Name>
    yields

        "collections":{
            "msgs":{
                "count":1
            },
            "priorities":{},
    ...
    

    You can also query all the queues in your MessageVPN and get the message count for each. Thanks @Ramesh Natarajan -RN !

  • sjaaksjaak Member Posts: 74 ✭✭✭

    Nice! Let me have a look. Thanks, @TomF :smile:

  • TomFTomF Member, Employee Posts: 246 Solace Employee

    Thank @Ramesh Natarajan -RN - he deserves the credit!

  • sjaaksjaak Member Posts: 74 ✭✭✭

    Thank you @Ramesh Natarajan -RN ! :smile: Will explore the solution.

  • sjaaksjaak Member Posts: 74 ✭✭✭

    @Ramesh Natarajan -RN @TomF This works! This info is crucial for us to manage and sell the Boomi-Solace platform to customers. Thank you! :smiley:

Sign In or Register to comment.