How can I replay from a specific message ID?

SeanTYH
SeanTYH Member Posts: 10 ✭✭
edited August 2023 in General Discussions #1

My main purpose is to be able to replay from a specific message. I want to be able to keep track of which was the last message that was successfully processed, so that I wont have to replay from the start or replay the whole day's worth of message. I think that in order to do this, I would need to be able to find out how to find out what was the last successfully processed message ID.

But then comes another thought: are the generated message IDs in HA groups/replication groups sequential/incremental? I would like to be able to replay from the message that was directly after the last successfully processed message. For example - if the last successfully read message id is: rmid1:236f8-31b0b535515-00000000-a24b7e32, would the next message have an increment of one in a specific part of the message id?

I have reviewed this article, but it did not contain the information I am looking for.

Tagged:

Answers

  • Linus
    Linus Member Posts: 8 ✭
    #2

    Have you looked at Last Value Queues ? https://docs.solace.com/Messaging/Guaranteed-Msg/Queues.htm#lvq

  • SeanTYH
    SeanTYH Member Posts: 10 ✭✭
    #3
    https://solace.community/discussion/comment/6361#Comment_6361

    Hello @Linus, thanks for your input. I just took a look at the Last Value Queue, and this key point caught my attention:

    "A client publishing Guaranteed messages can apply a topic subscription to a Last Value Queue so that it attracts all the messages that the client publishes. This allows the client to use the Last Value Queue to accurately determine the very last Guaranteed message that it successfully published."

    My "service" application in question is only doing subscription, as it creates a queue using JcsmpFactory then using consumerFlowProperties, replays and processes the messages. This seems to be contradictory as I m not doing any publishing. Will it still fit in this use case?

    Your suggestion is still useful as I can use the last value queues to modify the adapter client applications that receives data and publishes it to my service application via solace.

  • Linus
    Linus Member Posts: 8 ✭
    #4

    Yeah I was confused I blame in on Monday. No it does not seem to for fill your use case. For consuming applications some external storage (db) seems like a resonable point for storing last processed message but I will let the Solace experts answer your question better.

  • SeanTYH
    SeanTYH Member Posts: 10 ✭✭
    edited August 2023 #5

    Leaving this as a comment so that I can illustrate the problem a bit better:

    My service application currently creates a queue using JcsmpFactory then using consumerFlowProperties, replays by sending the contents of the replay to a queue and then I have a JmsListener that listens to that queue to process the messages.

    So when my adapter application sends messages to the event broker, it is added to both the replay log and the queue. Say the replay log contains message A, and the adapter sends message B to message F and the service goes down having processed some messages. I then bring the service up and I want to replay, but only from the last processed message. Lets say the last processed message is D, how can I determine that I replay E and F?

    Here is some simplified source code that I have that illustrates the above example: https://stackoverflow.com/questions/76666897/does-consumerflowproperties-from-solace-carryover

Categories

This Month's Leaders

This Week's Leaders