Moving messages from deadletter queue to main queue in Solace
I am new to Solace and using Boomi to push messages to Solace. I tried JMS Toolbox to view the messages. Do we have any tools or any options to move the messages from the Dead Letter queue to Main queue. I am looking for any of the options available. Can you please guide me in right direction please.
Comments
-
We use Boomi and Solace as well. That's a good question.
- We have built Boomi processes to do that. So, get messages from DMQ and send them back into the original queue. The risk here is that, with Boomi, a GET does not support guaranteed delivery. This means that, if the Boomi process fails, you will lose your messages.
- As far as I know, JMS toolbox does not support moving messages. You can copy messages from the DMQ to the original queue. But you then need to remove them as well. This is OK for a few messages, but not for big numbers.
It's a pity that the Solace UI doesn't support this, because you always need a separate tool to manage queues. And JMS Toolbox isn't the sexiest tool out there. Customers don't like the UI of JMS Toolbox.
IMO, it would be great if the Solace Cloud UI would support his. Just a simple move messages function would do the trick here. As well as an option to view payloads!
2 -
Hi @sjaak ,
I agree that JMSToolbox isn't the sexiest of tools for sure! And thanks for the excellent feedback that you would like to see this within the UI.@Poornima,
You can go the JMSToolbox route if you're looking to do this manually but if you're trying to automate this, or write some code to check for certain conditions and then route back to the original queue (or maybe somewhere else for manual inspection) it's good to keep in mind that a Solace DMQ is just a regular queue. You can use any of our APIs (or integrations like Boomi) to listen on the DMQ, write your processing and then publish where you'd like.1 -
@Poornima, have you considered using HermesJMS? I believe it has the ability to move messages between queues.
There are a couple of points to consider here:
1. The broker doesn't allow PubSub+ Manager to access message contents. Doing so would violate a security principle: admin users administer the broker, NOT the message data.
2. As @marc pointed out, moving messages from a DMQ to another queue should not be automated. If a message is on a DMQ, it's the messaging architecture's way of saying "something is really broken here. I need you to help fix it!"Some examples of where moving a message from a DMQ to a normal queue makes sense:
1. The contents of the message broke the consumer (caused it to crash) - the so called poison message case. You've updated the consumer to handle the poison message, deployed the new code and now you need to process the previously poison messages. There is no order dependency in the messages;
2. Your consumer was temporarily over loaded and messages were not processed in time and so expired to the DMQ. You've scaled up the consumer to deal with this load in future, validated that the messages still need to be processed, and know that there is no order dependency.Some examples of where moving a message from a DMQ to a normal queue should not happen:
1. The messages need to be processed in order;
2. Your consumer didn't know what to do with the message so just didn't acknowledge it;
3. The DMQ is being used for business logic exception processing.I hope that helps!
2 -
hi @TomF ,
Thank you for your feedback. The reason we don't use HermesJMS is that it is no longer supported and maintained. We are a Boomi partner for 5 years and for a few months also a Solace partner.In the Boomi world, this is how the average DevOps team is dealing with on a daily basis:
- View payloads so they can be copied to a Boomi process to reproduce the issue using the Boomi test mode (step debugger)
- Copy messages from DMQ to the original queue (poison messages) after the root cause in the Boomi integration process has been fixed so they will be re-processed again
As far as I know, JMSToolbox is the only tool out there. The feedback we got so far during pre-sales meetings
- Prospects and customers are surprised about the poor UI of JMSToolbox. Versus the nice and clean UI of Solace itself. They expect the above features to be part of Solace itself. As a Solace partner, we experience this as a competitive disadvantage for Solace.
- If security principles stand in the way of implementing the suggested features, that can be solved with additional roles.
- Relying on only one open-source tool such as JMXToolbox is a risk for Solace customers. What if it no longer works?
- How do other customers deal with DevOps support?
Sjaak
0 -
An update on this discussion
We bumped into an issue today with JMS Toolbox. See https://solace.community/discussion/925/how-to-retain-original-message-properties-like-topic-and-sender-id-when-a-message-is-moved-to-a-dmq
It appears that it changes Solace message headers when you move messages from queue A to B. For example, SenderID and Destination. JMS Toolbox empties the SenderID, and it overwrites the Destination header with the queue name, which overwrites the original topic name.
Challenges
- We can't route messages internally in Boomi using Solace headers when using JMSToolbox to move messages from a DMQ to the original queue. FYI: the destination (topic) is used to determine from the source system.
- Boomi customers cannot use JMSToolbox themselves to move messages to the original queue. As far as I know, there's no workaround for JMSToolbox. See previous comments.
To make a long story short: it would be great to add the following feature to the backlog: Move messages from queue A to B using the Solace Cloud UI. That will make a lot of Boomi and other low-code customers happy
0 -
Hey @sjaak , thanks for the update! Yeah, there is currently no way to just "move a message back" from the broker tooling itself. Since JMSToolbox is just a JMS client (not a broker tool), it's essentially creating a copy of the message and republishing it, which is why the Destination changes. Timestamps could also change as well.
Our Replay feature allows the replay of a series of messages from the replay log to any one particular queue, and it maintains all of the original message properties (such as Destination). But you can't replay a single message.
I'll raise the issue with our Product Line Mgmt team, and make sure this is on their radar.
1
