Solace Spring Cloud processor app issues when upgrading to version 4.2.0 from 3.4.0

MickeyZ · December 18

Our client is using Solace PubSub+ broker version 9.25 and are not going to move to the version >10.2, therefore we have to get our application working with the version 9.25 of broker.

Previously we used Java 8 and Spring Boot 2 and "com.solace.spring.cloud:spring-cloud-starter-stream-solace:3.4.0".
The config for processor was:

spring:

  cloud:

    function:

      # receiveSecurities if message is found to be an array of securities

      definition: receiveSecurity

    stream:

      binders:

        local-solace:

          type: solace

          environment:

            solace:

              java:

                host: tcp://x.x.x.x:nnnn

                msgVpn: default 

                clientUsername: default 

                clientPassword: default 

                connectRetries: 3

                connectRetriesPerHost: 0

                reconnectRetries: 3

      bindings:

      #The bindings section is used to define your input and output channels.

        receiveSecurity-in-0:

          destination: client.hostName.securities

          #The presence of "group" tells the binder to follow the "consumer group" pattern; if not present the "publish-subscribe" pattern will be used.

          group: hostName

          consumer:

            concurrency: 5

      solace:

        bindings:

          # receiveSecurities-in-0 if message is found to be an array of securities

          receiveSecurity-in-0:

            consumer:

              #Queue should not be created by the application

              provisionDurableQueue: false

              provisionErrorQueue: false

              provisionSubscriptionsToDurableQueue: false

              #Avoid the scst prefix etc

              queueNamePrefix: ""

              useFamiliarityInQueueName: false

              useDestinationEncodingInQueueName: false

              useGroupNameInQueueName: false

The publisher was publishing to:

spring
  cloud
    solace
      bindings:

      #The bindings section is used to define your input and output channels. 

        sendSecurity-out-0:

          #Publishes to a TOPIC 

          destination: client/hostName/securities

          #destination: scst/wk/trdapps/plain/cibc/trdapps/securities

          binder: local-solace

and inside the Solace GUI we have set for:

the queue: client.hostName.securities

the topic: client/hostName/securities

All this was was working well, untill we moved to Java version 17 and Spring Boot version 3.

The client configuration does not work any more.

The following versions can not connect to the broker:

"com.solace.spring.cloud:spring-cloud-starter-stream-solace:3.4.0"
"com.solace.spring.cloud:spring-cloud-starter-stream-solace:5.0.0"
"com.solace.spring.cloud:spring-cloud-starter-stream-solace:5.4.0"

The only version that it could connect is:

"com.solace.spring.cloud:spring-cloud-starter-stream-solace:4.2.0"

Our aim is still to connect to the same queue, as client is the one who chooses the name of the queue. However, using the same config as above I am getting following errors:

[main] c.s.s.c.s.b.p.SolaceQueueProvisioner: Creating durable queue scst/wk/hostName/plain/client.hostName.securities for consumer group hostName

[main] c.s.s.c.s.b.p.SolaceQueueProvisioner: Durable queue provisioning is disabled, scst/wk/hostName/plain/clent.hostName.securities will not be provisioned nor will its configuration be validated

[main] c.s.s.c.s.b.p.SolaceQueueProvisioner: Testing consumer flow connection to queue scst/wk/hostName/plain/client.hostName.securities (will not start it)

[Context_2_ReactorThread] c.s.c.u.LogWrapper: Client-1: Got BIND ('scst/wk/hostName/plain/client.hostName.securities') Error Response (503) - Unknown Queue

[main] c.s.s.c.s.b.p.SolaceQueueProvisioner: Failed to connect test consumer flow to queue scst/wk/hostName/plain/nc.client.hostName.securities. Provisioning is disabled, queue was not provisioned nor was its configuration validated.

JCSMPErrorResponseException: 503: Unknown Queue

How can I disable that prefix "scst/wk/hostName/plain/" appearing in the queue name?
I alreaday used disabling of the prefix elements:

      solace:

        bindings:

          # receiveSecurities-in-0 if message is found to be an array of securities

          receiveSecurity-in-0:

            consumer:

              #Queue should not be created by the application

              provisionDurableQueue: false

              provisionErrorQueue: false

              provisionSubscriptionsToDurableQueue: false

              #Avoid the scst prefix etc

              queueNamePrefix: ""

              useFamiliarityInQueueName: false

              useDestinationEncodingInQueueName: false

              useGroupNameInQueueName: false

mpatel · December 18

Can you try setting the queueNameExpression=destination,

solace:

        bindings:

          # receiveSecurities-in-0 if message is found to be an array of securities

          receiveSecurity-in-0:

            consumer:
              queueNameExpression=destination

Just FYI default value for queueNameExpression is as below.

Ref: https://github.com/SolaceProducts/solace-spring-cloud/blob/9b07a576f31491c23b0e2a71d3548a20b38c1a6e/solace-spring-cloud-stream-binder/solace-spring-cloud-stream-binder-core/src/main/java/com/solace/spring/cloud/stream/binder/properties/SolaceConsumerProperties.java#L62

	/**
	 * A SpEL expression for creating the consumer group’s queue name.
	 * Modifying this can cause naming conflicts between the queue names of consumer groups.
	 * While the default SpEL expression will consistently return a value adhering to <<Generated Queue Name Syntax>>,
	 * directly using the SpEL expression string is not supported. The default value for this config option is subject to change without notice.
	 */
	private String queueNameExpression = "'scst/' + (isAnonymous ? 'an/' : 'wk/') + (group?.trim() + '/') + 'plain/' + destination.trim().replaceAll('[*>]', '_')";

MickeyZ · December 19

Thanks very much @mpatel.
Definitely an improvement. There is another issue though.

INFO  [main] c.s.s.c.s.b.p.SolaceQueueProvisioner: Subscribing queue client.hostName.securities to topic client.hostName.securities

ERROR [main] o.s.c.s.b.BindingService: Failed to create consumer binding; retrying in 30 seconds

It is trying to connect to the queue client.hostName.securities which is correct, but the topic is client.hostName.securities
In the Solace dashboard for the queue client.hostName.securities we have Subsciptions topic:

client/hostName/securities

In my config file I have:

spring:

cloud:

stream:

bindings:

receiveSecurity-in-0:

destination: client.hostName.securities

So it is pointing to subscirbing queue and in turn that queue is subscribed to the topic:
client/hostName/securities

Do I need to change anything in my config, so that it points to the topic

client/hostName/securities

instead of:
client.hostName.securities

mpatel · December 19

In your config I see provisionSubscriptionsToDurableQueue: false. There is no such property.

The correct property name is addDestinationAsSubscriptionToQueue: false

Please see the documentation here > https://github.com/SolaceProducts/solace-spring-cloud/tree/stage-3.2.0/solace-spring-cloud-starters/solace-spring-cloud-stream-starter#solace-consumer-properties

if you set addDestinationAsSubscriptionToQueue: false, you issue should be resolved.

MickeyZ · December 20

Thanks very much @mpatel.

That change worked for me.

Now I have in my config 2 lines for error queue:

autoBindErrorQueue: false
provisionErrorQueue: false

Do I need both of them?

MickeyZ · 2:30AM

I have another question regarding the case when the Solace queue is not available.

Our microservices were not able to connect to their respective Solace queues. Although it seems they were able to connect to Solace, the queues themselves were not available, causing a huge volume of time outs. An example of the error is included below.
At the moment their configuration has a set number of retries for the connection to Solace, but no provision for what should occur when a queue is unavailable. The result in this instance was a near endless loop of timeouts that caused the log files to grow to 20+ terabytes.
Although the queue unavailability is a client side issue, we need to have a contingency in place to prevent this from happening in future.

Please could you advise how can we prevent this indefinite attempts to connect to the queue that is not available.

The example of the error is bellow:

2024-10-09 09:52:14,438 Wlace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener [pool-9-thread-4] Received error while trying to read message from endpoint PUBS.TYPE.SECURITY.PRICE

com.solacesystems.jcsmp.StaleSessionException: Tried to call receive on a stopped message consumer.

	at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.throwClosedException(FlowHandleImpl.java:2042)

	at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:920)

	at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:887)

	at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:326)

	at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:258)

	at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.receive(InboundXMLMessageListener.java:123)

	at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.run(InboundXMLMessageListener.java:96)

	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

	at java.util.concurrent.FutureTask.run(FutureTask.java:266)

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

	at java.lang.Thread.run(Thread.java:750)

Caused by: com.solacesystems.jcsmp.JCSMPTransportException: (JCSMPTransportException) Error communicating with the router. (KeepAlive)

	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2493)

	... 4 common frames omitted

Caused by: com.solacesystems.jcsmp.JCSMPTransportException: (Client name: client.hostname.com/2038009/00420001   Local port: -1   Remote addr: x.x.x.x  Remote port: nnnnn) - Timeout happened when reading response from the router.

	at com.solacesystems.jcsmp.protocol.impl.TcpChannel.executePostOnce(TcpChannel.java:243)

	at com.solacesystems.jcsmp.protocol.impl.ChannelOpStrategyClient.performOpen(ChannelOpStrategyClient.java:99)

	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.performOpenSingle(TcpClientChannel.java:426)

	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.access$800(TcpClientChannel.java:122)

	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2329)

	... 4 common frames omitted

Caused by: java.net.SocketTimeoutException: null

	at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:129)

	at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.open(SimpleSmfClient.java:905)

	at com.solacesystems.jcsmp.protocol.smf.ZSmfClient.open(ZSmfClient.java:62)

	at com.solacesystems.jcsmp.protocol.smf.SSLSmfClient.open(SSLSmfClient.java:90)

	at com.solacesystems.jcsmp.protocol.smf.SSLSmfClient.doPostNoResponse(SSLSmfClient.java:241)

	at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.doPost(SimpleSmfClient.java:235)

	at com.solacesystems.jcsmp.protocol.impl.TcpChannel.executePostOnce(TcpChannel.java:199)

	... 8 common frames omitted

🎄 Happy Holidays! 🥳

Happy Holidays!

Solace Spring Cloud processor app issues when upgrading to version 4.2.0 from 3.4.0

Comments

Categories

This Month's Leaders

This Week's Leaders