Error occurred for Solace queue depth request message: 0 while reading from Solace using dataflow
I am trying to stream messages from Solace to BigQuery using DataFlow where I am using the pubsub+ IO connector for Apache Beam provided in the below github repo. https://github.com/SolaceProducts/solace-apache-beam
I am able to read the messages but also getting below errors which is due to failures in reading the backlogs from the queue. Based on the queue depth dataflow workers need to scale up which is NOT happening. The user account I am using has subscribe access in the queue. I checked with the Solace team and "Show SEMP Queue" is enabled, Solace team is able to make below SEMP queries too in the same queue and vpn.
warning: Error Response (403) - Publish ACL Denied - Topic '#SEMP/<host router>/SHOW'
Error: Error occurred for Solace queue depth request message: 0
Notice: Exception occurred in message producer notification handler
Notice: Failed to execute message bus SEMP request, restarting Requestor: Timeout occurred performing request
Error: Encountered a JCSMPException querying queue depth
Error: getSplitBacklogBytes() unable to read bytes from: q/<the solace queue name>
Just to read the queue depth why the user ACL need publish access? There may be many consumers to the queue where for security purpose they will not be given publisher access. As per my analysis the error is occurring from MsgBusSempUtil.java. function -> queryRouter() in the below line:
return requestor.request(requestMsg, 5000, sempShowTopic); where requestMsg is BytesXMLMessage having query string attached as : String.format("<rpc><show><queue><name>%s</name><vpn-name>%s</vpn-name></queue></show></rpc>", queueName, vpnName) and sempShowTopic: final String sempShowTopicString = String.format("#SEMP/%s/SHOW", routerName); sempShowTopic = JCSMPFactory.onlyInstance().createTopic(sempShowTopicString);
Please help how to resolve this issue. Thanks in advance.