Metrics for Queue monitoring in Solace

Sameer · July 2021

Hello Team,

We are trying to explore the APIs to get the metrics like TOTAL_MESSAGES_SENT, TOTAL_MESSAGES_RECEIVED ,etc for the queue. We have gone through the documentation and tried to use the packages (from solace.messaging.utils.manageable import ApiMetrics, Metric) mentioned in the documentation.

We are trying to use the to_string_api_metrics method to get the relevant stats about the messages from queue.

When we tried to run this we are getting some static numbers which are not getting updated if we are pushing or consuming any messages.
Sample output :-
[ALL]: {"TOTAL_BYTES_SENT": 0, "TOTAL_MESSAGES_SENT": 0, "PUBLISHER_WOULD_BLOCK": 0, "DIRECT_BYTES_SENT": 0, "DIRECT_MESSAGES_SENT": 0, "PERSISTENT_BYTES_SENT": 0, "NONPERSISTE
NT_BYTES_SENT": 0, "PERSISTENT_MESSAGES_SENT": 0, "NONPERSISTENT_MESSAGES_SENT": 0, "PERSISTENT_MESSAGES_REDELIVERED": 0, "NONPERSISTENT_MESSAGES_REDELIVERED": 0, "PERSISTENT_BYTES_REDELI
VERED": 0, "NONPERSISTENT_BYTES_REDELIVERED": 0, "PUBLISHER_ACKNOWLEDGEMENT_RECEIVED": 0, "PUBLISHER_WINDOW_CLOSED": 0, "PUBLISHER_ACKNOWLEDGEMENT_TIMEOUTS": 0, "CONTROL_MESSAGES_SENT": 3
, "CONTROL_BYTES_SENT": 581, "CONNECTION_ATTEMPTS": 1, "PUBLISHED_MESSAGES_ACKNOWLEDGED": 0, "PUBLISH_MESSAGES_DISCARDED": 0, "DIRECT_BYTES_RECEIVED": 0, "DIRECT_MESSAGES_RECEIVED": 0, "B
ROKER_DISCARD_NOTIFICATIONS_RECEIVED": 0, "UNKNOWN_PARAMETER_MESSAGES_DISCARDED": 0, "TOO_BIG_MESSAGES_DISCARDED": 0, "PERSISTENT_ACKNOWLEDGE_SENT": 0, "PERSISTENT_DUPLICATE_MESSAGES_DISC
ARDED": 0, "PERSISTENT_NO_MATCHING_FLOW_MESSAGES_DISCARDED": 0, "PERSISTENT_OUT_OF_ORDER_MESSAGES_DISCARDED": 0, "PERSISTENT_BYTES_RECEIVED": 0, "PERSISTENT_MESSAGES_RECEIVED": 0, "NONPER
SISTENT_BYTES_RECEIVED": 0, "NONPERSISTENT_MESSAGES_RECEIVED": 0, "CONTROL_MESSAGES_RECEIVED": 3, "CONTROL_BYTES_RECEIVED": 604, "TOTAL_BYTES_RECEIVED": 0, "TOTAL_MESSAGES_RECEIVED": 0, "
COMPRESSED_BYTES_RECEIVED": 0}

We are using Python in our Project.

Could you please suggest on this.

Aaron · July 2021

Hi there @Sameer . The metrics that you're currently looking at are the client API metrics for that one specific connection, these are gathered by the message API during the lifetime of that connection. I believe you probably want broker metrics, stats and data about the Solace broker, queues on it, and stuff like that. You'll need to check out our SEMP management API for that.... fully RESTful, can use for monitoring and configuration (e.g. make a new queue). You access SEMP using your management port and credentials, the same as when you log in to the PubSub+Manager GUI.

Swagger docs: https://docs.solace.com/API-Developer-Online-Ref-Documentation/swagger-ui/monitor/index.html

Or specifically: https://docs.solace.com/API-Developer-Online-Ref-Documentation/swagger-ui/monitor/index.html#/queue/getMsgVpnQueue

Something like:

$ curl -u admin:admin http://192.168.42.35:8080/SEMP/v2/monitor/msgVpns/default/queues/q1

{
    "data":{
        "accessType":"exclusive",
        "alreadyBoundBindFailureCount":0,
        "averageRxByteRate":0,
        "averageRxMsgRate":0,
        "averageTxByteRate":0,
        "averageTxMsgRate":0,
        "bindRequestCount":0,
        "bindSuccessCount":0,
        "bindTimeForwardingMode":"store-and-forward",
<SNIP>

Also, BTW, the Python API is still fairly new and features are still being added to it, so I'm not sure if all the session statistics are fully implemented yet. Perhaps why you were seeing some unchanging values..?

Sameer · July 2021

Hello @Aaron . Thanks for the reply.

I guess the values were not changing or static even after I pushed the messages due to the fact that these are message API which are not related to broker metrics (as you mentioned) .

I understand what you meant and the APIs you suggested looks like an answer I was after. Do you have a sample code regarding the REST APIs which I can refer to ? I tried some github links but only few were using SEMP Apis.

Regards
Sameer Garg

Aaron · July 2021

I don't think we have a lot of tutorials/samples on using the SEMPv2 API. It's meant to be "self-documenting", but I also think there is a LOT of info there that is confusing at first.

One of the easier ways to learn the API, or at least become familiar/knowledgeable about what it is capable of, is knowing that everything you can do in the PubSub+ Manager GUI, you can also do using SEMP:

The GUI uses SEMP "under the covers" to communicate with the broker. So if you navigate point-and-click around the Manager GUI, and you find the info you're looking for, that can help you "zoom in" to the appropriate section of the SEMP docs.

Doing a bit of Googling, I found this as well, might help? https://pypi.org/project/solace-semp-monitor/

Sameer · July 2021

Thanks for response @Aaron . I really appreciate the response.

It would have been better if I get hold of some documentation or sample code.

Regards
Sameer Garg

Sameer · July 2021

Hey @Aaron ,

I tried to hit the APIs using the postman tool.
Steps I followed :-
1. I created the API token from the Solace and use it for the authentication purpose.
2. Given the permission Get My Services, Create Services, Delete My Services.
3. Saved the token as variable apiToken and value(inital and current value) with the token I recieved in step 2.
4. Use a get request for the URL https://##############:>portno>/about (tried other as well)
5. I am getting the below error

405

<![CDATA[Method Not Allowed]]>

<![CDATA[
Method not allowed
]]>

1:1628

I tried changing to post request as well. I can see only Secured REST Host exposed in my Free account so I am using https protocol.

Can you please help me here. I am not sure what I am doing wrong .

Thanks in advance.

Regards
Sameer Garg

ChristianHoltfurth · July 2021

Hi Sameer,
Was good to chat with you earlier! I just wanted to follow up and close the loop here in the forum.

As we discussed, there are multiple API endpoints (Cloud Console, Broker Admin, Client API) and multiple levels in Solace that you can get stats on (Service/broker, message vpn, queues, client connections etc pp).

The API token you mention above is for the Solace Cloud Console, which gives you access to listing your services/brokers etc, but not the deep level stats you are looking for in this case, that's why you are seeing "Method not allowed" as this method you are calling isn't implemented there.
To get the queue stats you are looking for, you'll have to point your Postman to the admin interface of the broker directly (usually the same DNS/IP as your client user if you are using a software broker) and use the SEMPv2 API that Aaron gave as an example above.

Now, if you are using a shared broker amongst many application teams, it's a recommendation to gather those stats once by a central monitoring tool, so that multiple application teams are not sending loads and loads of stats requests to the brokers all asking for the same (or similar) stats (this could have a negative effect on broker performance in extreme cases, if you are polling many detailed stats every second). So it's best to coordinate the gathering of these stats in bigger deployments and shared environments.
Your organisation is already providing such a central monitoring and statistic gathering service and it would be best to get your stats from there as we discussed.

Best regards,
Christian

Metrics for Queue monitoring in Solace

Comments

Categories

This Month's Leaders

This Week's Leaders