🎄 Happy Holidays! 🥳

Most of Solace is closed December 24–January 1 so our employees can spend time with their families. We will re-open Thursday, January 2, 2024. Please expect slower response times during this period and open a support ticket for anything needing immediate assistance.

Happy Holidays!

Please note: most of Solace is closed December 25–January 2, and will re-open Tuesday, January 3, 2023.

Defragmentation Failing: Unmovable Local Transaction

AllenW
AllenW Member Posts: 18 ✭✭✭
in PubSub+ Event Broker #1

I'm having trouble running defragmentation of a broker's message spool - which is currently at 90% and using up all the disk space.

Each time I run defragmentation it completes about 5% until it exits with the same Exit Condition:
```
Last Result:
Completed On: Oct 30 2023 07:29:21 UTC
Completion %: 5%
Exit Condition: Unmovable Local Transaction
Owned by Client Name: xxx-client-0001/82/#003f000
```
I have located the client/queue and forced a client disconnect as per Solace documentation: https://docs.solace.com/Messaging/Guaranteed-Msg/Defragmenting-Guaranteed-Messaging.htm
"In some cases committing or rolling back a transaction may not be possible if the transaction is not in a prepared state. In these cases, a client disconnect may be required to resolve the blocked transactions."

Are there any other approaches I can take to allow defrag to occur? Thanks

Tagged:

Best Answer

  • AllenW
    AllenW Member Posts: 18 ✭✭✭
    edited November 2023 #2 Answer ✓

    Thanks Ron & Marc.

    We were able to find a work-around to this issue. Thought I would mention here for posterity.

    In the documentation there is an example of an exit condition 'Unmovable XA Transaction XID:XXX-XXXXXXX' which in that case, allows us to either roll-back or delete the transaction blocking the defragmentation using the XID.

    But in this case where it was an 'Unmovable Local Transaction' where only the client is listed. Given the client was a producer we had to ask the application owners to temporarily disconnect the client from their end. We attempted to disconnect the client from the Solace side, but it would automatically reconnect instantly - as it was configured to do. Only after this we were able to execute defragmentation and clear disk space.

    solace> enable
    solace# admin
    solace(admin)# system message-spool
    solace(admin/system/message-spool)# defragment-spool-files start

    While this resolved our disk space issue defragmentation was still unable to complete fully - with it stopping halfway through, this time with the exit code: 'Unmovable XA Transaction XID:XXX-XXXXXXX'. So, this required rolling-back the particular transaction causing the defrag to exit.

    solace> enable
    solace# admin
    solace(admin)# message-spool message-vpn <vpn-name>
    solace(admin/message-spool)# rollback-transaction xid <xid>

    Then we were able to complete defrag.

    This broker is using Solace Version 10.4.1.76

Answers

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 963 admin
    #3

    Hi @AllenW,

    Unfortunately I've never dealt with this so I won't be any help, but I wanted to chime in and make sure you've also opened a support ticket on this.

  • RonInglesby
    RonInglesby Member, Employee Posts: 12 Solace Employee
    #4

    Hi @AllenW

    I will ask in our Slack if there is another approach rather than disconnecting the client. In the meantime can you please create a support ticket and email me the details (ron.inglesby@solace.com).

  • AllenW
    AllenW Member Posts: 18 ✭✭✭
    edited November 2023 #5 Answer ✓

    Thanks Ron & Marc.

    We were able to find a work-around to this issue. Thought I would mention here for posterity.

    In the documentation there is an example of an exit condition 'Unmovable XA Transaction XID:XXX-XXXXXXX' which in that case, allows us to either roll-back or delete the transaction blocking the defragmentation using the XID.

    But in this case where it was an 'Unmovable Local Transaction' where only the client is listed. Given the client was a producer we had to ask the application owners to temporarily disconnect the client from their end. We attempted to disconnect the client from the Solace side, but it would automatically reconnect instantly - as it was configured to do. Only after this we were able to execute defragmentation and clear disk space.

    solace> enable
    solace# admin
    solace(admin)# system message-spool
    solace(admin/system/message-spool)# defragment-spool-files start

    While this resolved our disk space issue defragmentation was still unable to complete fully - with it stopping halfway through, this time with the exit code: 'Unmovable XA Transaction XID:XXX-XXXXXXX'. So, this required rolling-back the particular transaction causing the defrag to exit.

    solace> enable
    solace# admin
    solace(admin)# message-spool message-vpn <vpn-name>
    solace(admin/message-spool)# rollback-transaction xid <xid>

    Then we were able to complete defrag.

    This broker is using Solace Version 10.4.1.76

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 963 admin
    #6

    Thank you for sharing @AllenW 🙌

Categories

This Month's Leaders

This Week's Leaders