How to access Solace Cloud statistics and integrate it with monitoring platforms (Datadog)

sjaak
sjaak Member Posts: 109 ✭✭✭
in General Discussions #1

Hi, I did some research regarding monitoring. Solace offers Insights which is running on Datadog. But it seems it's a closed system. We want to know how we can access Solace Cloud statistics and send them to public cloud monitoring platforms like Datadog. So we can integrate Solace with existing monitoring solutions of customers and our own monitoring platform we use for Boomi. Is this on the roadmap? We also got this question from several prospects.

Tagged:

Comments

  • Tamimi
    Tamimi Member, Administrator, Employee Posts: 538 admin
    #2

    For collecting broker metrics and send them to a custom monitoring platform, you can use syslog or SEMP to query any of the metrics you are interested in. You can check out more details about syslog forwarding here https://docs.solace.com/Best-Practices/Monitoring-BP.htm which will be very helpful for your usecase.

    As of right now, Insights for metrics only go to Insights/datadog so as a short term solution, you can either consider the syslog forwarding and using SEMP as mentioned earlier or use PubSub+ Monitor. Le me know if this is helpful and does what you were looking for

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    edited January 2021 #3

    Hi @Tamimi ,
    Thank you for your reply. SEMP should work I think. But it's a lot of work to set it up: 1 HTTP calls per metric + parsing. Since Solace Insights is already using Datadog, I wonder if there's documentation available on how Datadog and Solace have been integrated. For example, with Boomi we setup 1 Datadog .yaml with all the metrics we want to get and Datadog takes care of the rest using JMX. So 1 config and 1 call. That's it.

  • TomF
    TomF Member, Employee Posts: 412 Solace Employee
    #4

    Hi @sjaak, I see what you mean. We don't have that kind of yaml file yet.

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    #5

    Question: how can we access the syslog of Solace Cloud instances? We want to upload them to Datadog and generate alerts from the logs

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    edited February 2021 #6

    Aha...I found a hidden feature in the Solace Cloud management console to forward the syslogs to and rsyslog server. You need to click on "Advanced options" on the management page before you see it :smile:

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    edited March 2021 #7

    @Tamimi We have 1 urgent metric we want to measure, and that the current number of messages in a queue. I checked the SEMP API's but I can't find an operation that exposes that information. Customers are chasing us for a solution because this is one of the key KPIs to monitor a message broker in our world. Which is integrating Cloud applications (Finance, CRM, Warehouse Management, HR) using persistent messaging. Any sales order which fails needs to trigger an alert mechanism. Hence Datadog but we need the ability the get to these metrics.

    Any ideas are welcome! :smile:

  • TomF
    TomF Member, Employee Posts: 412 Solace Employee
    #8

    @sjaak As far as I can see, what you would need to do is use the monitor SEMP API to get the message meta-data for every message (bear in mind pagination) and programmatically count them.

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    #9

    @TomF Looking at the available operations, this is unworkable IMO. I was hoping that there's an operation for this which simply returns a count with 1 call. Is that on the roadmap?

  • TomF
    TomF Member, Employee Posts: 412 Solace Employee
    #10

    Hi @sjaak, success! My colleague @Ramesh Natarajan -RN has pointed out that you can actually get the message count from the monitor API:
    GET http://localhost:8080/SEMP/v2/monitor/msgVpns/default/queues/<Queue Name>
    yields

        "collections":{
        "msgs":{
            "count":1
        },
        "priorities":{},
...

    You can also query all the queues in your MessageVPN and get the message count for each. Thanks @Ramesh Natarajan -RN !

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    #11

    Nice! Let me have a look. Thanks, @TomF :smile:

  • TomF
    TomF Member, Employee Posts: 412 Solace Employee
    #12

    Thank @Ramesh Natarajan -RN - he deserves the credit!

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    #13

    Thank you @Ramesh Natarajan -RN ! :smile: Will explore the solution.

  • sjaak
    sjaak Member Posts: 109 ✭✭✭
    #14

    @Ramesh Natarajan -RN @TomF This works! This info is crucial for us to manage and sell the Boomi-Solace platform to customers. Thank you! :smiley:

Categories

This Month's Leaders

This Week's Leaders