solace spark monitoring

Hi @nayan_sharma, I'm sorry to be pedantic but there are some things I need to check: when you say your streaming job is consuming, is your job using a subscription, or is it consuming from a queue? This is important as there's no real concept of a topic as something on the broker you can query. We can still help, though!
The key concept here is that we have a management and monitoring API called SEMP which will help you. Using this, you can query objects such as queues and clients and return the number of messages consumed and all that good stuff. You can even use your existing session to query SEMP using messages - this is called SEMP over the message bus.
For queues the answer is therefore pretty simple. If you're just using subscribers, you'll need to get statistics from the client (subscriber) itself.

Ah! That makes life a lot easier then. Using SEMP v2 you can query a queue:
curl -u <username:password> -X GET http://<broker IP:port>/SEMP/v2/monitor/msgVpns/<msgVpn Name>/queues/<queue name>

Will return JSON with all the statistics gathered about the queue:

{
"data":{
"accessType":"exclusive",
"alreadyBoundBindFailureCount":0,
"averageRxByteRate":0,
"averageRxMsgRate":0,
"averageTxByteRate":0,
"averageTxMsgRate":0,
...

There's a lot of error counters that might be of interest, for instance maxMsgSpoolUsageExceededDiscardedMsgCounttells you how many messages were rejected by the queue because the spool use has been exceeded, while maxTtlExpired... counters tell you how many messages have expired off the queue due to TTL being exceeded. bindSuccessCount tells you how many consumers are currently consuming from the queue.

You can also query statistics on the individual flows from a queue - that is an open consumption "channel" from a queue to a client, using URI:
SEMP/v2/monitor/msgVpns/<msgVpn name>/queues/<queue name>/txFlows/<flow id>.
Get the flow IDs from SEMP/v2/monitor/msgVpns/default/queues/test/txFlows