How do avoid 403: Replication Is Standby ?

Options
vadivelan
vadivelan Member Posts: 3 ✭
in General Discussions #1

We are using “spring-cloud-starter-stream-solace” to consume the message from queue but receiving the below error frequently. Please refer the below details and help us to avoid the issue.



application.yml configuration:

binders:

           <<bindername>>:

              type:

               environment:

                 solace:

                    java:

                       clientUsername: <<User Name>>

                       connectRetries: 3

                       connectRetriesPerHost: 0

                       reconnectRetries: 3

                       host: tcps://<<Host Name 1>>:55443,tcps:// <<Host Name 2>>:55443

                       msgVpn: <<VPN Name>>

solace:

           bindings:

              <<function name>>-in-0:

                 consumer:

                    queueAdditionalSubscriptions: <<topic name>>

                    provisionDurableQueue: true

                    queueNamePrefix: ''

                    useFamiliarityInQueueName: false

                    useDestinationEncodingInQueueName: false

                    useGroupNameInQueueName: false



Error Message

com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.receive - Received error while trying to read message from endpoint <<Queue Name>>

com.solacesystems.jcsmp.StaleSessionException: Tried to call receive on a stopped message consumer.

             at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.throwClosedException(FlowHandleImpl.java:1957)

             at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:899)

             at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:866)

             at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:279)

             at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:211)

             at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.receive(InboundXMLMessageListener.java:93)

             at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.run(InboundXMLMessageListener.java:73)

             at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

             at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

             at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

             at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

             at java.base/java.lang.Thread.run(Thread.java:834)

Caused by: com.solacesystems.jcsmp.JCSMPTransportException: (JCSMPTransportException) Error communicating with the router. (KeepAlive)

             at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2438)

             ... 4 common frames omitted

Caused by: com.solacesystems.jcsmp.JCSMPErrorResponseException: 403: Replication Is Standby

             at com.solacesystems.jcsmp.protocol.impl.TcpChannel.executePostOnce(TcpChannel.java:232)

             at com.solacesystems.jcsmp.protocol.impl.ChannelOpStrategyClient.performOpen(ChannelOpStrategyClient.java:97)

             at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.performOpenSingle(TcpClientChannel.java:418)

             at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.access$800(TcpClientChannel.java:114)

             at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2274)

             ... 4 common frames omitted

Answers

  • vadivelan
    vadivelan Member Posts: 3 ✭
    #2
    Options

    Hi @JamilAhmed, Need your support to resolve the issue.

  • JamilAhmed
    JamilAhmed Member, Employee Posts: 17 Solace Employee
    #3
    Options

    Hi @vadivelan,

    I just tested in my environment with some brokers that are in a DR replication setup and have reproduced the error.

    There are two parts here:

    1. The reconnect parameters you've set that govern how long the services wait for the broker connection to get restored. (You're essentially giving up too quickly.)
    2. Excessive logging when the service gives up connecting but the binder continues to do some work

    For point (1), the settings and resulting behaviour is the same as what @Kaliappans hit in an earlier issue. @marc recommended to adjust them or perhaps even take them out entirely and rely on the defaults instead: https://solace.community/discussion/comment/5306#Comment_5306

    For point (2), looks like this has been seen before too and is already tracked in a Jira issue: https://github.com/SolaceProducts/solace-spring-cloud/issues/174. While previously it was reported in the context of a High-Availability switch, your case involves a switch between Replication mates. We'll make sure to cover off all these situations in the behaviour improvement for it. (fyi: @giri, @amackenzie)

    Thanks

    Jamil

Categories

This Month's Leaders

This Week's Leaders