Client Data Messages count is increasing though I sent just one message from source broker

Sumit
Sumit Member Posts: 9
edited September 2019 in Connectors & Integrations #1

I see my SolaceIOConnector as consumer in my queue now.
I sent one message but my destination broker message vpn stats are something like this

Message VPN Stats
Client Data Messages (msgs) Received 155 Sent 154 and ......growing
Incoming Message Discards 1

Why is this count increasing? - After stopping the connector, these messages stopped. Please help me understand this behavior or this is a bug in the stats.
Why is the one message in discard?

Queue Stats
Incoming Messages Not Queued
No Local Delivery 13,608

Bridges
red_to_blue
Client Data Messages Received 1 Sent 1

Thank you!!

Answers

  • [Deleted User]
    [Deleted User] Posts: 0 ✭✭

    The SolaceIO is performing backlog checks. It sends a SEMP request over the message backbone to check queue backlog bytes. This should be about a message a sec per queue.
    For discard check with "show stats clients detailed" to see what the ingress discard is. If it is No Subscription match, then do a "show log no-subscription-match" to see what topic was not subscribed to.

  • Sumit
    Sumit Member Posts: 9

    Thanks for the reply KenBarr.
    These backlog check messages should be part of control flow and not the data flow. Isn't it a bug as its giving wrong report on the UI.
    Also, I didn't get the second part. How can I check "show stats clients detailed" and then "show log no-subscription-match" if required. Please explain.
    Thank you!

  • Sumit
    Sumit Member Posts: 9
    edited September 2019 #4

    Between this is the current state of the connector that shows there was no message passed to it from the queue.

    Starting Solace session [solaceioconnector1-sumitc-092411/22/#002a0001/EeWdfQlEc8] on queue[blue_test_red_test_Queue]...
    .
    Encountered a Parser Exception querying queue depth: java.lang.NullPointerException
    .
    Stats for Queue [blue_test_red_test_Queue] : {"queueBacklog":0,"emptyPolls":296,"messagesReceived":0,"messagesCheckpointReady":0,"messagesCheckpointComplete":0}"

    Is this the "emptyPolls" you were referring to the backlog checks above?
    Thank you!

  • Sumit
    Sumit Member Posts: 9
    edited September 2019 #5

    After some more testing I found that the queue name that I used to start solacesink was the one used as bridge queue and I was listening to the same queue in the solaceio. So this setup has issues where client sends message to queue directly and wants to consume it from the queue. Is this correct?

    I changed it to have topic instead of queue in solacesink, subscribed topic to bridge queue in source vpn and then subscribed the same topic to app_queue on the destination vpn. I see messages are flowing to solaceio now but I am not able to print payload. Its empty. What am I missing?

    Also, app_queue is showing "Failure Queue Disabled" as 4 in queue stats.
    Bridge stats is showing 4 messages received but none sent.
    No message discards anywhere.

    .apply("ExtractLines", ParDo.of(new DoFn<SolaceTextRecord, String>() {
                        @ProcessElement
                        public void processElement(@Element SolaceTextRecord line, OutputReceiver<String> receiver) throws Exception {
    LOG.info("msg received >>>>>>>>>>>>>>>> "+line.getMessageId()+" | payload >>>> "+line.getPayload() +" | destination >>>> "+line.getDestination());
    LOG.info("msg string >>>>> " + line.toString());
    receiver.output(line.getPayload());
    }
    }));
    

    Output in logs

    line.getMessageId() >> 792046
    line.getPayload() >> no output
    line.getDestination() >> solacetest
    line.toString() >> org.apache.beam.sdk.io.solace.SolaceTextRecord@d51bf03f

    Thanks!

  • [Deleted User]
    [Deleted User] Posts: 0 ✭✭

    Control messages are things like connections and subscription adds. A SEMP over the message bus query is treated like a data message that the broker internal client is handling. May be a bit confusing in the wrap up view, but not a bug.

    These backlog check messages should be part of control flow and not the data flow. Isn't it a bug as its giving wrong report on the UI.

  • [Deleted User]
    [Deleted User] Posts: 0 ✭✭

    queueBacklog:0 means there is zero bytes of data in the queue the UnboundedSolaceReader is bound to.
    emptyPolls: 296 means in the last 2 minutes the UnboundedSolaceReader.advance() method timed out 296 waiting for a message to arrive and returned false.

    {"queueBacklog":0,"emptyPolls":296,"messagesReceived":0,"messagesCheckpointReady":0,"messagesCheckpointComplete":0}"

    Is this the "emptyPolls" you were referring to the backlog checks above?
    Thank you!

  • Sumit
    Sumit Member Posts: 9

    Thanks for clarifying the two points above. After fixing the issue I can see that the correct count is being shown under the stats of the queue so I think it should be ok for the message vpn stats to have it in the data messages. Probably some more documentation around the stats at different levels will help.
    Couple of things left in my previous post:
    1 I am kind of block on getting the empty payload, what am I doing wrong here in the code?
    2 What is failed queue disabled? Didnt find any documentation around it. Why do I see 4 messages in there?
    3 Why bridge stats is not showing the message that I can see in SolaceIO?

  • [Deleted User]
    [Deleted User] Posts: 0 ✭✭

    @Sumit said:
    Thanks for clarifying the two points above. After fixing the issue I can see that the correct count is being shown under the stats of the queue so I think it should be ok for the message vpn stats to have it in the data messages. Probably some more documentation around the stats at different levels will help.
    Couple of things left in my previous post:
    1 I am kind of block on getting the empty payload, what am I doing wrong here in the code?

    For empty payload. How is message being sent? Is the message being produced via something like a JMS text message or JCSMP mapped message? There are examples of how to read each we just need to match up the data to how you are consuming.

    2 What is failed queue disabled? Didn't find any documentation around it. Why do I see 4 messages in there?

    Is this seen on broker discard stats? If so it is likely the queue received 4 messages but receive side of queue was shut.

    3 Why bridge stats is not showing the message that I can see in SolaceIO?

    What do you mean by this. I assume you are receiving messages over the bridge and forwarding to SolaceIO. Are they coming over bridge direct or persistent?

  • Sumit
    Sumit Member Posts: 9

    Thanks KenBarr.
    1 This is kafka message. What extra is required here to get payload? Code snippet is provided above.
    2 This is under queue stats in "Incoming Messages Not Queued"
    3 Under bridge stats I can see the messages as received but nothing is coming as sent though I can see the message flowing into the SolaceIO. Please see above where I shared the message id printed in the output logs.