Get all messages from a queue

Vempati
Vempati Member Posts: 7 ✭
edited November 2021 in PubSub+ Event Broker #1

Dear Team,

I have a requirement to retrieve all the messages from a queue at a given point of time. I believe On-demand JMS or Consume operations can only read one message per request. Do we have any parameter to configure in JMS or Consume operation in Mulesoft Soalce connector.

Can we achieve this via SEMP apis? I would like to hear any other better or feasible approaches for this requirement.

Design:

Mule API with Get All messages(Rest endpoint)-> Solace connectivity via JMS or solace connector or SEMP-> Connect to Queue and read all the messages.

Thanks,
Vempati

Tagged:

Answers

  • amackenzie
    amackenzie Member, Employee Posts: 268 Solace Employee
    #2

    Hello,
    There is no way to get more than 1 message at a time in any Solace API and within the context of the Solace MuleSoft connector (Solace has no ability to change the JMS connector as it's provided by MuleSoft) there is not a concept of batches of messages in our consume operation.

    The use case is somewhat problematic too in that "at a given point in time" implies that you would have a dynamic batch size (assuming we added batches) that is changed at runtime per batch. There is nothing built into the connector that would provide queue depth.

    As far as SEMP being used... SEMP is a management API and has NO access to message payloads so it would not be appropriate for this use case.

  • Vempati
    Vempati Member Posts: 7 ✭
    #3

    Hi,

    Thanks for the confirmation. Yes it is dynamic batches to read from a queue. So, If I understand correctly there is no solution?

    Thanks,
    Vempati

  • amackenzie
    amackenzie Member, Employee Posts: 268 Solace Employee
    #4

    I am not aware of a solution. I suppose you could use a sub flow to consume messages in a loop and store them in some cache. If there is a break in the publishing such that the queue has a static number of messages for "a while" (at least the time it takes to process the current messages), then the loop could be run until there are no more messages which breaks the loop.
    Then in your main flow, operate off the cache?

    Does Mulesoft support this use case with Anypoint MQ? It sounds like a database use case not ideally suited to a message queue.

  • Vempati
    Vempati Member Posts: 7 ✭
    #5

    Yes, I could think of same approach but then the below Issues might encounter.

    1. On each call from consume operation, does it do bind and unbind? or the session persists for some time? Will it be a problem from soalce Infra , If the consume operations called multiple times in a loop?
    2. It could fail , If there is a time out from front end application while we are processing in mulesoft.
    3. Error handling would be much difficult. Depending on the load.

    I don't think it is even possible with AMQ, I totally agree Ideal case should be from a db but here we need process this data from downstream applications and put in to queue

  • amackenzie
    amackenzie Member, Employee Posts: 268 Solace Employee
    #6
    1. There is a new jcsmp flow created for each time Consume is called. However, the connection itself is reused. Connections are much more expensive than flows.
  • Vempati
    Vempati Member Posts: 7 ✭
    #7

    okay, thanks for the confirmation. If the same connection is being used then atleast to some extent it is good.

  • Mayur Patil
    Mayur Patil Member Posts: 1 ✭
    #8
    Vempati: I have the same project requirement. Did u find any solution on "to retrieve all the messages from a queue at a given point of time". Kindly let me know if you got something otherwise, I would have to develop Java code and integrate with Mule flow.

Categories

This Month's Leaders

This Week's Leaders