Detect which services are alive

radekm

Hi,

we have several services which use Solace to communicate.
We also want to notify users if some service is not running.
To do that every service publishes a heartbeat
to a topic service-name/heartbeat and when the heartbeat stops
we assume that the service is not running.

This is working, but I'm wondering if there is a better/easier/more natural solution?
Because Solace broker itself must monitor which sessions are connected
so if we could somehow get this information or notification
when a new session is connected or some existing session disconnected
we could simplify our implementation.

Thanks

TomF

Hi @radekm, an interesting question. I think the answer is that you need to keep your heartbeat solution.

The long answer is: yes the broker will tell you when a client stops responding. The broker and client share heartbeats (keepalives) depending on the protocol you're using. When there's a keepalive failure, the broker will disconnect the client, and send a notification via Syslog (CLIENT_CLIENT_DISCONNECT, if I remember correctly). So you could easily set up syslog rules to deal with this.

BUT... and it's a big but: this all operates at the TCP and API layer. Provided your process/JVM etc is still running, the API processing will continue and API keepalives will still flow. What this won't tell you is if your application code is stuck somewhere: say it's processing a loop, got stuck and can't allocate any more threads. In this case, you need your application code to send a keepalive message - which is what you're already doing.

Now, if you hadn't already created a heartbeat mechanism I would say use the client disconnection notice as a good approximation for application level heartbeats. However, you've already implemented the most thorough solution: my advice is to stick with it now you've already done the hard work.

TomF

Hi @radekm, an interesting question. I think the answer is that you need to keep your heartbeat solution.

The long answer is: yes the broker will tell you when a client stops responding. The broker and client share heartbeats (keepalives) depending on the protocol you're using. When there's a keepalive failure, the broker will disconnect the client, and send a notification via Syslog (CLIENT_CLIENT_DISCONNECT, if I remember correctly). So you could easily set up syslog rules to deal with this.

BUT... and it's a big but: this all operates at the TCP and API layer. Provided your process/JVM etc is still running, the API processing will continue and API keepalives will still flow. What this won't tell you is if your application code is stuck somewhere: say it's processing a loop, got stuck and can't allocate any more threads. In this case, you need your application code to send a keepalive message - which is what you're already doing.

Now, if you hadn't already created a heartbeat mechanism I would say use the client disconnection notice as a good approximation for application level heartbeats. However, you've already implemented the most thorough solution: my advice is to stick with it now you've already done the hard work.