SSL/TLS connection rejected by remote host
Below error is triggered and fills the log file event and system log files. Please assist what could be the issue.
2021-08-25T09:46:47.432+00:00 <local3.info> **** SOLACEINFO: SYSTEM: SYSTEM_SSL_CONNECTION_REJECTED: - - SSL Connection rejected: reason (Connection closed by remote host); connection to 127.0.0.1:8091 from 127.0.0.1:56656
Above client-username is #client-username mapped to default profiles.
replaced the VPN name to ****
One of my analysis:
127.0.0.1 with port 8091 is trying to connect as per the service configuration on the broker MQTT port from 8000 is disabled for SSL. Wil this be an issue ? and why all of sudden the alert (INFO) is triggered ?
MQTT WEB N - - Mgmt Test 8000 D D Disabled at Vpn
Comments
-
Hello @Velu ! Welcome to the Community.
Ok, so there's a couple things. First, you can't use the
#client-username
username. That is for internal use (along with anything else you see starting with "#"). Use thedefault
client-username, or make a new one.From your CLI output, I'll copy in the column headers as it makes it easier to explain:
Flags Legend: TP - Transport T+U - TCP and UDP S - SSL (Y=Yes, N=No, -=not-applicable) C - Compressed (Y=Yes, N=No, -=not-applicable) R - Routing Ctrl (Y=Yes, N=No, -=not-applicable) VRF - VRF (Mgmt=management, MsgBB=msg-backbone) A - Admin State (U=Up, D=Down, -=not-applicable) O - Oper State (U=Up, D=Down, -=not-applicable) Status Service TP S C R VRF MsgVpn Port A O Failed Reason ---------- --- ----- ----- --------------- ----- --- -------------------------- MQTT WEB N - - Mgmt Test 8000 D D Disabled at Vpn
- the "N" means it is a non-SSL port, so you can't connect using SSL
- the "D D" means that the port is down as well, both configured and operational status
- the reason is because it has been disabled at the VPN level
- and I'm not sure why you're trying to connect to port 8091 ..?
So, if you want an SSL connection, you'll probably want to connect to port 8883 (SSL) or 8443 (WSS)... but those are just for the
default
VPN. If you are connecting to another Message VPN, then you'll have to figure out what ports are configured. E.g.:show message-vpn <BLAH> service
Finally, you'll need to ensure you see "U U" for admin and operational state. You'll need to ensure you have copied a server certificate onto the broker so that it can support SSL connections. https://docs.solace.com/Configuring-and-Managing/Managing-Server-Certs.htm
Good luck!
0 -
@Aaron,
Thank you for the welcome and the brief.Am not sure why it picks 8091 port? Is there any configuration explicit to get this system based service ?
Is that by enabling MQTT for SSL address the issue ?
Flags Legend:
TP - Transport
S - SSL (Y=Yes, N=No, -=not-applicable)
C - Compressed (Y=Yes, N=No, -=not-applicable)
VRF - VRF (Mgmt=management, MsgBB=msg-backbone)
A - Admin State (U=Up, D=Down, -=not-applicable)
O - Oper State (U=Up, D=Down, -=not-applicable)---Press any key to continue, or `q' to quit---
Service TP S C VRF MsgVpn Port A O Failed ReasonSMF TCP N N MsgBB 55555 D D Disabled at vpn
SMF TCP N Y MsgBB 55003 D D Disabled at vpn
SMF TCP Y N MsgBB 55443 U U
REST WEB N - MsgBB Test 9000 D D Disabled at Vpn
REST WEB Y - MsgBB Test 9443 U U
SMF WEB N - MsgBB 8008 D D Disabled at vpn
SMF WEB Y - MsgBB 1443 U U
MQTT TCP N - MsgBB Test 1883 D D Disabled at Vpn
MQTT TCP Y - MsgBB Test 8883 U U
MQTT WEB N - MsgBB Test 8000 D D Disabled at Vpn
MQTT WEB Y - MsgBB Test 8443 U U
AMQP TCP Y - MsgBB U D Not Permitted
AMQP TCP N - MsgBB Test 5672 D D Disabled at Vpn
AMQP TCP Y - MsgBB Test 5671 U U0 -
Hi again @Velu . Ok, from your whole output, I see a lot of "U U", which means "admin up" and "operational up". So I guess you have a server certificate successfully installed!
From this line:
Service TP S C VRF MsgVpn Port A O Failed Reason ---------- --- --- ----- --------------- ----- --- -------------------------- MQTT WEB Y - MsgBB Test 8443 U U
So if you're looking to use WebSockets to connect (e.g. from JavaScript or something), then you'll want port 8443, and that will connect you to your "Test" Message VPN.
And I can tell that you're on a hardware appliance, rather than a software broker, so make sure you are using the message bus IP address, not the management one.
Finally, just an FYI for formatting: you can post blocks of code/text inside three backticks ```. Makes it a lot easier to read CLI output! 👍
0 -
Hi @Aaron,
The broker is running on Kubernetes. We have disabled SMF ports. Clients are using only REST API's and SSL cert auth is mandatory.
How does 8091 is picked by the internal process.
Any config changes ? like any web ports to be enabled.( I would like to modify the port (8091) to the REST SSL port (9443). is it possible ?).
Please suggest.
(Surely will adopt the usage/formatting ).
0 -
Hi @Velu Yes you can change port number, For REST you can use below to specify port.
solace(configure)# message-vpn
solace(configure/message-vpn)# service rest
solace(configure/message-vpn/service/rest)# incoming
solace(...ure/message-vpn/service/rest/incoming)# listen-port [ssl]0 -
Thank you for the cli @Abhishek
@AaronTrying to understand, that the port 8091 was not configured for the service on the broker, but the #client-username picks it ?
client details : #client-username
Client: #client Type: Internal Client Profile: #client-profile ACL Profile: #acl-profile Authorization Group: Originating IP Address: 127.0.0.1 Subscriptions: 6 No Local Delivery: No Eliding Enabled: No Eliding Topics: Current: 0 High Water Mark: 0 Deliver To One: Local Priority: 1 Network Priority: 1 Client Id: 4 Message VPN: Test Uptime: 41d 13h 6m 24s Scheduled Disconnect Time: N/A Slow Subscriber: No Client Username: #client-username Original Client Username: #client-username User: SolOS Description: Internal Message Bus Software Version: 7.14.0.9 Software Date: Jul 20 2020 09:45:34 Platform: Linux26-x86_64_opt - C SDK ---Press any key to continue, or `q' to quit--- One Shot Events: Large Message: not raised Message Too Big: not raised Parse Error: not raised Max Eliding Topics: not raised Total Ingress Flows: 0 Total Egress Flows: 0 Web Transport Session: N/A SSL Version: N/A SSL Cipher: N/A SSL Downgraded to Plain Text: No
0 -
Hi again @Velu ... ok, so I think the confusion here is that you're looking at an "internal use only" client that the broker maintains whenever a Message VPN is enabled.
#client
is not your client connection. It is for the broker's own internal communication. That's why you're seeing a weird port. Note the uptime is 41 days! 😅As mentioned in my first response, ignore anything that starts with
#
as they are all for internal stuff. Check this post: https://solace.community/discussion/356/what-are-these-objects-that-start-withSo, what you need to do to get connected to your broker:
- in your Message VPN, create a new client-username to use. Configure however you want for authentication. Maybe "basic none" is the best way to start.
- If you've made a new client-profile for your client-username, and you want to use Guaranteed messaging, please make sure you enable a bunch of message spool settings in that profile
If you've made a new acl-profile for your client-username, you'll need to change the default behaviour of everything to "allow". It is set to disallow by default.
- Easier to just use the
default
acl-profile
- Easier to just use the
Finally, as a test, make sure you can connect to the broker with just a regular Solace (SMF) client to make sure your username/password/whatever is working. Either SdkPerf, or the PubSub+ Manager "TryMe!" test applications.
Hope that helps!
0 -
@Aaron,
Trying to deploy a test broker on Kube, to find out the configurations that are related. I will revert with the findings/analysis from the new deployment. Also with the current environment, couldn't prolong the manipulations since clients are active. mystery as of now0 -
hi all,
I remember seeing this log entry repeatedly in a customer environment (GKE Anthos) last year, and it was something about load balancer heartbeat. Though it was not port 8091 that time.
For K8s based deployment, the port exposed to outside world could be different from the ports shown in the service, since the LB will define those external port and map it back to the Pod's ports.
For my case last time, it was found out to be something with the GKE environment and the support from GKE helped the client out, I didn't get what was the root cause etc.
0