StaleSessionException spam when disconnected from host
Situation:
Using spring cloud stream with a solace binder. Connection to host dropped.
Current behavior:
After all attempts to reconnect failed, the application starts producing tons of logs with StaleSessionException (which took over all disk space on the server over night).
Expected behavior:
Application shutdown.
Tried to do:
Manipulating reconnect properties did not yield any result. Disabled reconnect with the following:
solace: java: connect-retries-per-host: 0 reconnect-retries: 0
The application did not make any reconnect attempts (which is expected) and went straight to exception spam. Right now the only option I see to avoid spam is setting those properties to -1 so that the application keeps trying to reconnect indefinitely. But in case of disconnect, I need the whole application to go down. Am I missing something?
Stack trace of the exception log:
2022-08-19 18:33:09,854 WARN [-11-thread-1] c.s.s.c.s.b.i.InboundXMLMessageListener: Received error while trying to read message from endpoint #P2P/QTMP/v:ldnpsol2005n01/scst/2a0a2991-0496-48b0-bffa-3431de5a2897 com.solacesystems.jcsmp.StaleSessionException: Tried to call receive on a stopped message consumer. at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.throwClosedException(FlowHandleImpl.java:1849) at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:793) at com.solacesystems.jcsmp.impl.flow.FlowHandleImpl.receive(FlowHandleImpl.java:760) at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:293) at com.solace.spring.cloud.stream.binder.util.FlowReceiverContainer.receive(FlowReceiverContainer.java:225) at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.receive(InboundXMLMessageListener.java:114) at com.solace.spring.cloud.stream.binder.inbound.InboundXMLMessageListener.run(InboundXMLMessageListener.java:91) at com.solace.spring.cloud.stream.binder.inbound.BasicInboundXMLMessageListener.run(BasicInboundXMLMessageListener.java:21) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: com.solacesystems.jcsmp.JCSMPTransportException: (JCSMPTransportException) Error receiving data from underlying connection. at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2299) ... 4 common frames omitted Caused by: com.solacesystems.jcsmp.JCSMPSecurityException: Error performing login to LoginContext (intranet.barcapint.com) at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.getKRBToken(SimpleSmfClient.java:1244) at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.open(SimpleSmfClient.java:839) at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.doPostNoResponse(SimpleSmfClient.java:305) at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.doPost(SimpleSmfClient.java:222) at com.solacesystems.jcsmp.protocol.impl.TcpChannel.executePostOnce(TcpChannel.java:206) at com.solacesystems.jcsmp.protocol.impl.ChannelOpStrategyClient.performOpen(ChannelOpStrategyClient.java:90) at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.performOpenSingle(TcpClientChannel.java:415) at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.access$800(TcpClientChannel.java:105) at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel$ClientChannelReconnect.call(TcpClientChannel.java:2135) ... 4 common frames omitted Caused by: javax.security.auth.login.LoginException: intranet.barcapint.com at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:788) at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:592) at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:747) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:672) at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:670) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:670) at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:581) at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.getKRBToken(SimpleSmfClient.java:1242) ... 12 common frames omitted Caused by: java.net.UnknownHostException: intranet.barcapint.com at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1509) at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1368) at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1302) at java.base/java.net.InetAddress.getByName(InetAddress.java:1252) at java.security.jgss/sun.security.krb5.internal.UDPClient.<init>(NetClient.java:186) at java.security.jgss/sun.security.krb5.internal.NetClient.getInstance(NetClient.java:44) at java.security.jgss/sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:406) at java.security.jgss/sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:369) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.security.jgss/sun.security.krb5.KdcComm.send(KdcComm.java:353) at java.security.jgss/sun.security.krb5.KdcComm.sendIfPossible(KdcComm.java:253) at java.security.jgss/sun.security.krb5.KdcComm.send(KdcComm.java:229) at java.security.jgss/sun.security.krb5.KdcComm.send(KdcComm.java:200) at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:345) at java.security.jgss/sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:498) at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:756) ... 20 common frames omitted
Libraries I'm using
com.solacesystems:sol-jcsmp:10.11.0
com.solace.spring.cloud:spring-cloud-starter-stream-solace:3.3.1
com.solace.spring.boot:solace-java-spring-boot-starter:4.2.2
Comments
-
+1 i created an github issue for this problem:
https://github.com/SolaceProducts/solace-spring-cloud/issues/174
0