Try PubSub+

OpenShift TCPS routes

andan02andan02 Member Posts: 11

I am able to connect to the ws endpoints in solace (running on openshift) with the nodejs samples. I am unable to connect to the tcp/tcps endpoints (55555 and 55443) with nodejs and Java samples. Can someone tell me what I need to do to get the Java or nodejs samples to connect to the tcp and tcps endpoints?

Comments

  • andan02andan02 Member Posts: 11

    Telnet to the tcp port works:
    telnet ytcp-solace-pubsub.blah-blah 80

    here is one test:
    bin/DirectSubscriber tcp://ytcp-solace-pubsub.blah-blah:80 default password
    DirectSubscriber initializing...
    --LOG-- 11:27:04.127 [main] INFO com.solac.jcsmp.proto.impl.TcpClientChannel - Connecting to host 'orig=tcp://ytcp-solace-pubsub.blah-blah:80, scheme=tcp://, host=ytcp-solace-pubsub.blah-blah, port=80' (host 1 of 1, smfclient 1, attempt 1 of 1, this_host_attempt: 1 of 6)
    --LOG-- 11:27:04.287 [main] INFO com.solac.jcsmp.proto.impl.TcpClientChannel - Connection attempt failed to host 'ytcp-solace-pubsub.blah-blah' ConnectException com.solacesystems.jcsmp.JCSMPTransportException: (Client name: pro-desk.local/54811/#000f0001/Y6Tz6MYY_j Local addr: 192.168.4.37 Local port: 59872 Remote addr: ytcp-solace-pubsub.blah-blah Remote port: 80) - Error communicating with the router. cause: java.io.IOException: Could not read valid SMF Header from network. found smf version=0 ((Client name: pro-desk.local/54811/#000f0001/Y6Tz6MYY_j Local addr: 192.168.4.37 Local port: 59872 Remote addr: ytcp-solace-pubsub.blah-blah Remote port: 80) - )

  • marcmarc Member, Administrator, Moderator, Employee Posts: 361 admin

    Hi @andan02,
    A few questions:
    1. Did you deploy the Solace Event Brokers into OpenShift yourself? Or if someone else did it do you know if they used the openshift quickstart to do so? https://github.com/SolaceProducts/pubsubplus-openshift-quickstart
    2. Are you running the samples inside of openshift or trying to connect externally?

    My guess would be that the ports/routes may need to be configured. Note that there is a Validating the Deployment section of the quickstart guide that might prove to be useful.

  • andan02andan02 Member Posts: 11
    1. yes - I ran the quickstart
    2. the samples are running external to the cluster. I have masked the full url in the output log above to have "blah-blah" as the host.
  • marcmarc Member, Administrator, Moderator, Employee Posts: 361 admin

    Hi @andan02 - I think the issue might be that you're not running it against port 55555 unless I'm missing where that is mapped to port 80. Did you try this bin/DirectSubscriber tcp://tcp-solace-pubsub.blah-blah:55555 default password ?

  • andan02andan02 Member Posts: 11

    port 80 (for this route) is mapped in openshift to port 55555. The error I get - from the above example - "Error communicating with the router. cause: java.io.IOException: Could not read valid SMF Header from network. found smf version=0"

  • andan02andan02 Member Posts: 11

    btw, the nodejs example (using ws) works fine using ws://ws-solace-pubsub.blah-blah:80 (tcp-web port). Rest also works over the tcp-rest port. This seems to be unique to the tcp-smf port for the service.

  • marcmarc Member, Administrator, Moderator, Employee Posts: 361 admin

    thanks for the additional info @andan02. I'm not sure of the solution off the top of my head but will see what I can do to help out!
    In the mean time were you able to successfully validate the deployment using this info?

    Specifically I would double check that the tcp-smf ports all line up so port 55555 to the container itself goes to the NodePort which lines up with the port you're trying to connect to on the load balancer.

  • andan02andan02 Member Posts: 11

    oc get statefulset,service,pods,pvc,pv --show-labels
    I0503 15:26:25.795322 86634 request.go:621] Throttling request took 1.188045213s, request: GET:https://c114-e.us-south.containers.cloud.ibm.com:30488/apis/operators.coreos.com/v1alpha1?timeout=32s
    NAME READY AGE LABELS
    statefulset.apps/my-release-pubsubplus-ha 3/3 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=pubsubplus-ha,helm.sh/chart=pubsubplus-ha-2.4.0

    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE LABELS
    service/my-release-pubsubplus-ha LoadBalancer 172.21.243.89 2222:32484/TCP,8080:30166/TCP,1943:30710/TCP,55555:32231/TCP,55003:30151/TCP,55443:30827/TCP,55556:32422/TCP,8008:30635/TCP,1443:30142/TCP,9000:31919/TCP,9443:32398/TCP,5672:32641/TCP,5671:31032/TCP,1883:31015/TCP,8883:30378/TCP,8000:31336/TCP,8443:32428/TCP 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=pubsubplus-ha,helm.sh/chart=pubsubplus-ha-2.4.0
    service/my-release-pubsubplus-ha-discovery ClusterIP None 8080/TCP,8741/TCP,8300/TCP,8301/TCP,8302/TCP 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=pubsubplus-ha,helm.sh/chart=pubsubplus-ha-2.4.0

    NAME READY STATUS RESTARTS AGE LABELS
    pod/my-release-pubsubplus-ha-0 1/1 Running 0 4d5h active=true,app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha,controller-revision-hash=my-release-pubsubplus-ha-7c5895cff6,statefulset.kubernetes.io/pod-name=my-release-pubsubplus-ha-0
    pod/my-release-pubsubplus-ha-1 1/1 Running 0 4d5h active=false,app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha,controller-revision-hash=my-release-pubsubplus-ha-7c5895cff6,statefulset.kubernetes.io/pod-name=my-release-pubsubplus-ha-1
    pod/my-release-pubsubplus-ha-2 1/1 Running 0 2d13h app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha,controller-revision-hash=my-release-pubsubplus-ha-7c5895cff6,statefulset.kubernetes.io/pod-name=my-release-pubsubplus-ha-2

    NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE LABELS
    persistentvolumeclaim/data-my-release-pubsubplus-ha-0 Bound pvc-99fe7366-694b-47d7-9449-84bd1f687ccc 30Gi RWO ibmc-vpc-block-10iops-tier 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha
    persistentvolumeclaim/data-my-release-pubsubplus-ha-1 Bound pvc-812bbec7-806c-441f-9a9c-d85ba4a63aca 30Gi RWO ibmc-vpc-block-10iops-tier 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha
    persistentvolumeclaim/data-my-release-pubsubplus-ha-2 Bound pvc-d76f6454-0901-4ea2-8abd-0c4f0fa978f3 30Gi RWO ibmc-vpc-block-10iops-tier 4d5h app.kubernetes.io/instance=my-release,app.kubernetes.io/name=pubsubplus-ha

  • andan02andan02 Member Posts: 11

    I checked the logs on the 3 pods. I could not see any logs of any successful or unsuccessful publish or subscription. I do not know how to monitor if a connection is taking place or not over tcp/tcps to the smf ports.

  • marcmarc Member, Administrator, Moderator, Employee Posts: 361 admin

    Hi @andan02,
    I got a bit of info from our support team that might help:

    What causes JCSMP to give this error: "cause: java.io.IOException: Could not read valid SMF Header from network." when trying to connect? Something is wrong with the received packet and the JCSMP API cannot decode it. What is most likely happening is the openshift route is routing to the wrong host/port or something is corrupting the network data.

    With this info I was actually able to reproduce the issue by pointing the sample at tcp://google.com:80 which makes me think that for some reason the smf traffic isn't being properly routed through to port 55555.

    MJD-MacBook-Pro.local:~/git/solace-samples-java-jcsmp/build/staged$ ./bin/DirectSubscriber tcp://google.com:80 default default
    DirectSubscriber initializing...
    --LOG-- 08:42:00.014 [main] INFO  com.solac.jcsmp.proto.impl.TcpClientChannel - Connecting to host 'orig=tcp://google.com:80, scheme=tcp://, host=google.com, port=80' (host 1 of 1, smfclient 1, attempt 1 of 1, this_host_attempt: 1 of 6)
    --LOG-- 08:42:00.093 [main] INFO  com.solac.jcsmp.proto.impl.TcpClientChannel - Connection attempt failed to host 'google.com' ConnectException com.solacesystems.jcsmp.JCSMPTransportException: (Client name: MJD-MacBook-Pro.local/63808/#000f0001/aoVc2QtIeh   Local addr: 192.168.1.25 Local port: 62037   Remote addr: google.com  Remote port: 80) - Error communicating with the router. cause: java.io.IOException: Could not read valid SMF Header from network. found smf version=0 ((Client name: MJD-MacBook-Pro.local/63808/#000f0001/aoVc2QtIeh   Local addr: 192.168.1.25 Local port: 62037   Remote addr: google.com  Remote port: 80) - )
    

    I can however help with this question:

    I do not know how to monitor if a connection is taking place or not over tcp/tcps to the smf ports.

    You can see if a client is connected a few ways...note that you likely won't see your client since it is failing to connect.
    1. In PubSub+ Manager you can see connected SMF clients by choosing your Message VPN and then choosing the "Clients" Menu on the left hand side, and choosing "Solace Clients"
    2. In the logs use the show log event command to see the event log. This log will tell you when a client connects, if you look for events that say CLIENT: CLIENT_CLIENT_CONNECT that tells you when a messaging client is connecting to the broker. It will look something like this:

    2021-05-03T18:37:37.780+00:00 <local3.info> b0e95afab69a event: CLIENT: CLIENT_CLIENT_CONNECT: default MJD-MacBook-Pro.local/16740/#000f0001/LOrqg4D3OS Client (4) MJD-MacBook-Pro.local/16740/#000
    f0001/LOrqg4D3OS username default OriginalClientUsername(default) WebSessionId (N/A) connected to 172.17.0.2:55555 from 172.17.0.1:64534 version(10.10.0) platform(Mac OS X-x86_64 (Java 11.0.4+11)
     - JCSMP SDK) SslVersion() SslCipher() APIuser('marcdipasquale' Computer: 'MJD-MacBook-Pro.local' Process ID: 16740) authScheme(Basic) authorizationGroup() clientProfile(default) ACLProfile(defau
    lt) SSLDowngradedToPlainText(No) SSLNegotiatedTo() SslRevocation(Not Checked), Capabilities(unbind-ack, bind-response-endpoint-error-id)
    

    For next steps in troubleshooting this would there by some way of doing packet capture to see what is arriving into the load balancer on the outside of OpenShift vs. what is actually getting to the pod itself? Or even seeing stats within OpenShift that maybe tells us that the request is actually traversing the Load Balancer to the NodePort to the Pod?

  • andan02andan02 Member Posts: 11

    I need to figure out how to monitor this ingres. I do not know how to do this yet. In your example above, of course google wont respond with a proper SMF version. Port 80 is what openshift uses for non-ssl comms for any route you create. 443 is used for any ssl route you create. The route is connected to port 55555 in the port 80 route and port 55443 in the port 443 route I have created

    Is there someone within Solace that has used the TCP or TCPs connection out of openshift? I have now seen others with this issue in other forums. It would be great to figure this out.

  • andan02andan02 Member Posts: 11

    I found a way to monitor the route traffic. When I curl the url for the tcp-smf on port 80 you will see a spike in the traffic graph. At 13:46 on the graph I attempted to run the java code against the same url and port number - no traffic was detected. Is there something specific about the java code that is looking for 55555 or 55443 and if it does not find it, then it fails?

  • andan02andan02 Member Posts: 11

    can you test the java sample with a port other than 55555 or 55443 to see if it is the code or the connection?

  • andan02andan02 Member Posts: 11

    when I ran the code with the tcps connection info on port 443 I get a different result. It appears that traffic is getting through, but ssl is failing.

    --LOG-- 13:54:31.952 [main] INFO com.solac.jcsmp.proto.impl.TcpClientChannel - Connection attempt failed to host 'ytcps-solace-pubsub.blah' ConnectException com.solacesystems.jcsmp.JCSMPTransportException: (Client name: pro-desk.local/77914/#000f0001/Ri222wdFr9 Local addr: 192.168.4.37 Local port: 61030 Remote addr: ytcps-solace-pubsub.blah Remote port: 443) - Error communicating with the router. cause: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure ((Client name: pro-desk.local/77914/#000f0001/Ri222wdFr9 Local addr: 192.168.4.37 Local port: 61030 Remote addr: ytcps-solace-pubsub.blah Remote port: 443) - )

Sign In or Register to comment.