Issue with replication mode: Messages stuck in "Pending Replication" state

kyriathar

Hello all,

I am trying to set up replication here between 2 nodes.

I tried to follow the instructions as mentioned in the Steps-for-Replication-Configuration page.

I tried to reach the replication state starting from both:

• kubernetes HA deployment
• fresh docker deployments (so not HA, just standalone instances (initially)).
  

I seem to always be getting to the same state where I see a message in a queue, on primary node as "Pending Replication" and then on the secondary node (stand-by) I can't see the message in the queue (expected based on what the primary node reports).

I am not really sure what to post in terms of helpful information as I cannot seem to be making much out of the logs. Could you please advice on what information I could be posting.

I can see my created message VPN showing
in the primary node as: Status: Up/ Replication: Active
in the secondary node as: Status: Standby/ Replication: Standby

In terms of show replication command I am getting:

In primary node:

Replication Interface:

Replication Mate:                  v:f5948d15ebb4

    Plain Text:                    172.17.0.3:55555


    Compressed:


    SSL:


SSL:

  Default Cipher Suite List:       Yes

  Cipher Suites:                   ECDHE-RSA-AES256-GCM-SHA384

                                   ECDHE-RSA-AES128-GCM-SHA256

  Trusted Common Names:


ConfigSync:

  Bridge:

    Admin State:                   Enabled

    State:                         up

    Authentication:

      Scheme:                      Basic

    Compressed:                    No

    SSL:                           No

    Message Spool:

      Window Size:                 65535

    Retry Delay:                   3

    SSL Server Certificate Validation:

      Enforce Trusted Common Name: No

      Maximum Chain Depth:         3

      Validate Certificate Dates:  Yes

      Validate Server Name:        Yes

In secondary node:

Replication Interface:

Replication Mate:                  v:f698e5bce89a

    Plain Text:                    172.17.0.2:55555


    Compressed:


    SSL:


SSL:

  Default Cipher Suite List:       Yes

  Cipher Suites:                   ECDHE-RSA-AES256-GCM-SHA384

                                   ECDHE-RSA-AES128-GCM-SHA256

  Trusted Common Names:


ConfigSync:

  Bridge:

    Admin State:                   Enabled

    State:                         up

    Authentication:

      Scheme:                      Basic

    Compressed:                    No

    SSL:                           No

    Message Spool:

      Window Size:                 65535

    Retry Delay:                   3

    SSL Server Certificate Validation:

      Enforce Trusted Common Name: No

      Maximum Chain Depth:         3

      Validate Certificate Dates:  Yes

      Validate Server Name:        Yes

Thank you

kyriathar

So I managed to get it working.

I think what I was missing was enabling message spooling in both nodes (although not 100 % sure), so:

enable

solace# configure

solace(configure)# hardware message-spool

solace(configure/hardware/message-spool)# no shutdown 

I put together a few quick-start commands which seem to be working, in case it might be helpful to someone (and also for feedback in case something stands out). Please note I used default Message VPN:

1. Enable message spooling (in both nodes):

enable
solace# configure
solace(configure)# hardware message-spool
solace(configure/hardware/message-spool)# no shutdown

2. Bridges (specify user for the bridge) (in both nodes):

solace(configure/message-vpn/replication)# bridge
solace(configure/message-vpn/replication/bridge)# authentication basic
solace(...plication/bridge/authentication/basic)# client-username <name> [password <password>]
(I went with client-username default)

3. Configure Mates (in both nodes):

solace(configure)# replication
solace(configure/replication)# mate
solace(configure/replication/mate)# virtual-router-name v:<take peer router name with running command show router-name in peer node>
solace(configure/replication/mate)# connect-via <IP of peer>:55555

4. Turn on replication (in both nodes, 1 of them need to be active, rest in standby mode):

I did this through the admin UI.

So enable and then set mode to active for primary node / standby for secondary. I left the default values to the rest (so for auth: Basic - user: default).

5. Create a replicated topic:

Again did this from the admin UI. So, under Replication/ Replicated topics, e.g. "aReplicatedTopic".

6. Queue creation.

Because we want to create the queue in primary node only and have it propagated in secondary, we need to turn on config-sync for this in both nodes.

enable
configure
config-sync
no shutdown

Then back on the admin UI create the Queue in primary node only (we can see that it is also created in secondary), and add a subscription to match the replicated topic above, "aReplicatedTopic" in this case.

And that should hopefully work.