DLQ/Error QUEUE - Spring cloud Stream

akg17
akg17 Member Posts: 76 ✭
in PubSub+ Event Broker #1

When We publish message it goes to Queue via added topic subscription , which could be consumes using listener's easy. (Message published to Topic, Consumed From queue )

But, Why there is a difference in DLQ publish, I don't think when the message is getting republished to Error Queue it is published to Topic Endpoint, Instead it directly goes to Queue.

Why is that so ?

I tried adding topic subscription of that error queue topic to another queue it does not get the copy of message.

Tagged:

Comments

  • giri
    giri Member, Administrator, Employee Posts: 116 admin
    #2

    Hi @akg17,

    When you get a chance, give it a read of the documentation that details this feature here: https://github.com/SolaceProducts/solace-spring-cloud/tree/master/solace-spring-cloud-starters/solace-spring-cloud-stream-starter#failed-consumer-message-error-handling

    Here are the key points:

    • DMQ is used to capture re-routed failed messages as a consequence of Solace PubSub+ messaging features such as TTL expiration or exceeding a message’s max redelivery count
    • An Error Queue can be provisioned for a particular consumer group by setting the autoBindErrorQueue consumer config option to true and is simply a named durable queue
    • An Error Queue captures re-routed messages that are successfully consumed from the message broker yet cannot be processed by the application - which means that the events are directly delivered to the Error Queue, not based on a subscription.

    I hope this helps.

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    edited April 2022 #3

    Hi @akg17,

    You are correct that messages get published directly the the error queue w/ the info being in @giri's post. I think if we went back several years to design this again we might consider publishing error messages to a topic, but sending errors to a specific queue is really the one use case where I do think it sometimes makes sense to not publish to a topic.

    That said, what are your options if you want to have multiple consumers be able to receive these messages that are currently going to the error queue?

    1. Don't actually publish your messages to the error queue. Instead handle the errors yourself and use StreamBridge to publish them to dynamic topics of your choice. The benefit of this route is that it's actually more flexible b/c it allows you to publish messages that have different error situations to different topics to be handled in different ways.
    2. If you still want to use the error queue provided by the binder there is another work around. Internally to the broker when a message is published to a queue it actually gets sent to a "Network Topic" first. These network topics live in a Solace reserved topic space. For queues the format of their network topics is: #P2P/QUE/<Queue_Name> So a work around is to have your apps listen to that topic and they will be able to receive their own copy of the error message. You can find the network topic for your queue in PubSub+ Manager under the Queue detail information. (It would also be findable via cli or SEMP in the queue details). Note that is a work around so don't go using this information in all of your apps. In practice Solace could decide to change it so the network topic changes for a queue as this is an internal implementation detail in a reserved space, but I think it's pretty safe to use as I believe it's been that way for over 10 years ;)

    Hope that helps!

    image.png


Categories

This Month's Leaders

This Week's Leaders