Issue with replication mode: Messages stuck in "Pending Replication" state

PubSub+ Event Broker
1

Hello all,

I am trying to set up replication here between 2 nodes.

I tried to follow the instructions as mentioned in the Steps-for-Replication-Configuration page.
I tried to reach the replication state starting from both:

  • kubernetes HA deployment
  • fresh docker deployments (so not HA, just standalone instances (initially)).

I seem to always be getting to the same state where I see a message in a queue, on primary node as " Pending Replication " and then on the secondary node (stand-by) I can’t see the message in the queue (expected based on what the primary node reports).
I am not really sure what to post in terms of helpful information as I cannot seem to be making much out of the logs. Could you please advice on what information I could be posting.
I can see my created message VPN showing
in the primary node as: Status: Up/ Replication: Active
in the secondary node as: Status: Standby/ Replication: Standby

In terms of show replication command I am getting:
In primary node:



Replication Interface:
Replication Mate:                  v:f5948d15ebb4
    Plain Text:                    172.17.0.3:55555

    Compressed:

    SSL:

SSL:
  Default Cipher Suite List:       Yes
  Cipher Suites:                   ECDHE-RSA-AES256-GCM-SHA384
                                   ECDHE-RSA-AES128-GCM-SHA256
  Trusted Common Names:

ConfigSync:
  Bridge:
    Admin State:                   Enabled
    State:                         up
    Authentication:
      Scheme:                      Basic
    Compressed:                    No
    SSL:                           No
    Message Spool:
      Window Size:                 65535
    Retry Delay:                   3
    SSL Server Certificate Validation:
      Enforce Trusted Common Name: No
      Maximum Chain Depth:         3
      Validate Certificate Dates:  Yes
      Validate Server Name:        Yes

In secondary node:


Replication Interface:
Replication Mate:                  v:f698e5bce89a
    Plain Text:                    172.17.0.2:55555

    Compressed:

    SSL:

SSL:
  Default Cipher Suite List:       Yes
  Cipher Suites:                   ECDHE-RSA-AES256-GCM-SHA384
                                   ECDHE-RSA-AES128-GCM-SHA256
  Trusted Common Names:

ConfigSync:
  Bridge:
    Admin State:                   Enabled
    State:                         up
    Authentication:
      Scheme:                      Basic
    Compressed:                    No
    SSL:                           No
    Message Spool:
      Window Size:                 65535
    Retry Delay:                   3
    SSL Server Certificate Validation:
      Enforce Trusted Common Name: No
      Maximum Chain Depth:         3
      Validate Certificate Dates:  Yes
      Validate Server Name:        Yes

Thank you

2

So I managed to get it working.
I think what I was missing was enabling message spooling in both nodes ( although not 100 % sure ), so:


enable
solace# configure
solace(configure)# hardware message-spool
solace(configure/hardware/message-spool)# no shutdown

I put together a few quick-start commands which seem to be working, in case it might be helpful to someone (and also for feedback in case something stands out). Please note I used default Message VPN :

  1. Enable message spooling (in both nodes):

enable
solace# configure
solace(configure)# hardware message-spool
solace(configure/hardware/message-spool)# no shutdown

  1. Bridges (specify user for the bridge) (in both nodes):

solace(configure/message-vpn/replication)# bridge
solace(configure/message-vpn/replication/bridge)# authentication basic
solace(…plication/bridge/authentication/basic)# client-username [password ]
(I went with client-username default)
3. Configure Mates (in both nodes):

solace(configure)# replication
solace(configure/replication)# mate
solace(configure/replication/mate)# virtual-router-name v:<take peer router name with running command show router-name in peer node >
solace(configure/replication/mate)# connect-via :55555
4. Turn on replication (in both nodes, 1 of them need to be active, rest in standby mode):

I did this through the admin UI.

So enable and then set mode to active for primary node / standby for secondary. I left the default values to the rest (so for auth: Basic - user: default).

  1. Create a replicated topic:

Again did this from the admin UI. So, under Replication/ Replicated topics , e.g. “aReplicatedTopic”.

  1. Queue creation.

Because we want to create the queue in primary node only and have it propagated in secondary, we need to turn on config-sync for this in both nodes.
enable
configure
config-sync
no shutdown

Then back on the admin UI create the Queue in primary node only (we can see that it is also created in secondary), and add a subscription to match the replicated topic above, “aReplicatedTopic” in this case.

And that should hopefully work.