solace spark monitoring

nayan_sharma

Hi All,
I am using spark streaming with solace. Data is consumed from solace using custom streaming job. Now I have to build a monitoring job which will publish stats around the topic i.e. number of subscriber, number of messages received with granularity and alerts if any goes wrong.
Can Anyone guide me the right direction it would be very helpful.
Thanks. Stay safe.

TomF

Hi @nayan_sharma, I'm sorry to be pedantic but there are some things I need to check: when you say your streaming job is consuming, is your job using a subscription, or is it consuming from a queue? This is important as there's no real concept of a topic as something on the broker you can query. We can still help, though!
The key concept here is that we have a management and monitoring API called SEMP which will help you. Using this, you can query objects such as queues and clients and return the number of messages consumed and all that good stuff. You can even use your existing session to query SEMP using messages - this is called SEMP over the message bus.
For queues the answer is therefore pretty simple. If you're just using subscribers, you'll need to get statistics from the client (subscriber) itself.

nayan_sharma

Hi @TomF spark job is consuming from queue itself.

TomF

Ah! That makes life a lot easier then. Using SEMP v2 you can query a queue:
curl -u <username:password> -X GET http://<broker IP:port>/SEMP/v2/monitor/msgVpns/<msgVpn Name>/queues/<queue name>

Will return JSON with all the statistics gathered about the queue:

{
"data":{
"accessType":"exclusive",
"alreadyBoundBindFailureCount":0,
"averageRxByteRate":0,
"averageRxMsgRate":0,
"averageTxByteRate":0,
"averageTxMsgRate":0,
...

There's a lot of error counters that might be of interest, for instance maxMsgSpoolUsageExceededDiscardedMsgCounttells you how many messages were rejected by the queue because the spool use has been exceeded, while maxTtlExpired... counters tell you how many messages have expired off the queue due to TTL being exceeded. bindSuccessCount tells you how many consumers are currently consuming from the queue.

You can also query statistics on the individual flows from a queue - that is an open consumption "channel" from a queue to a client, using URI:
SEMP/v2/monitor/msgVpns/<msgVpn name>/queues/<queue name>/txFlows/<flow id>.
Get the flow IDs from SEMP/v2/monitor/msgVpns/default/queues/test/txFlows

nayan_sharma

@TomF Thank you for your excellent suggestion. I have gone through https://docs.solace.com/Best-Practices/Gathering-Stats-SEMP.htm#Queue and have build monitoring tool.