Introducing the "copy-message" command
Exciting news, Solace developers and administrators! You can now copy messages between queues!
PubSub+ Event Broker version 10.0.0 has a new management command, copy-message, that lets a broker administrator copy a message from the replay log to a queue, or from one queue to another (like from a DMQ to a client’s queue).
Why is this exciting? Well, when using guaranteed messaging, messages aren’t ever supposed to get lost. But in the real world, things aren’t always that simple. Your consumer might be offline for an extended period of time, causing time-to-live (TTL) to expire for the message. Or there might be an application issue preventing the app from processing the message, resulting in max-redeliveries being exceeded for the message. There could even be a subtle bug in the app, where it acknowledges receipt of the message just a bit too early, before it has fully completed processing of the message (for example, a database write that gets cached in RAM but doesn’t get flushed to disk before a power failure).
And let’s not forget operator error, where an administrator accidentally deletes the wrong message from the queue.
As long as the message is still available in either a dead-message-queue (DMQ) or the Replay log (you are enabling replay for your important message topics, right?), you can now easily get it back. The copy-message command lets you copy the message back to your consumer’s queue!
Copy-message is supported in CLI and SEMPv1 in PubSub+ Event Broker version 10.0.0 and later, and will be supported in PubSub+ Manager and SEMPv2 later this year.
Hopefully, you won’t need this command too often. But if you ever need to restore a message, it’s a great tool to have in your toolkit.
Comments
-
Thanks! Generally, broker production releases are delivered in Solace Cloud about a month after the self-managed brokers are released (we need a bit of time to integrate a new release into Solace Cloud). 10.0.0 was a preview release which won't be in Solace Cloud, but 10.0.1 will be a production release planned for mid-June, which means the release should be available in Solace Cloud in mid to late July.
2 -
@stevebu thanks for sharing that great news about a copy.
I want to give you feedback to get into the reality problems customers faces which will show this util need to be extended.
1.) Currently it is just to copy a single message
I would like to see a possibility to:
- Copy all
- Copy all message in certain time range (e.g. messages stranded today there) - the time range filter i would like to see a swell on purge or delete message command. Imagine you have messages in DMQ and they could not get resolved and so you want to just remove them after n days as no reason to resend
2.) The documentation of copy command does not really tell about handling of DMQ Eligible
As it mentions this use case i hope it changed DMQ Eligible back to "true". As i turns from "true" to "false" on move to DMQ. Honestly i never understood why that is done. It avoids what some people if thought of to set-up DMQ with main queue as Dead Letter to have an automated way every e.g. 1 hour to push messages back from DMQ to main queue. (without any need of copy)
3.) Solace must clarify how the delivery-count is handled in a copy back from DMQ to main queue.
I see 2 options. The delivery-count increases further where it was before it ended in DMQ.
So if there was a 10 retry you reach DMQ with delivery-count 10 and on next push and end again in DMQ you would have already 20. Or you even introduce an additional dmq-delivery-count
0
