How to publish and consume an attachment(.txt) file along with each message in queue?

Leo stany
Leo stany Member Posts: 3

Is there a way to publish a message and an attachment(.txt) file to a queue? Can this message and the attached file be consumed using java interface?

Best Answer

Answers

  • amackenzie
    amackenzie Member, Employee Posts: 260 Solace Employee
    #3 Answer ✓

    Hi Leo, thanks for the question.
    It really comes down to what you mean by "attachment". In PubSub+ you can have messages published that have a binary attachment. The contents of a binary attachment are up to you, so it certainly could be a serialized .txt file. The consuming applications would need to be aware of the format of that binary attachment in the message payload in order to know to deserialize to a .txt file. This can be done in a number of ways. A common way is publishing to a certain topic (which could then be subscribed to by the queue for persistence) and have your consumer applications consume from that queue. The message headers could be interrogated to see what topic the messages were published to and confirm that you should treat them as .txt in your consumer logic.
    Some links to concepts I mention are:
    https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Adding-Data-Payloads.htm
    https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Publishing-Guaranteed-Messages.htm
    https://docs.solace.com/PubSub-Basics/Core-Concepts-Endpoints-Queues.htm#topic-queue-mapping

  • Leo stany
    Leo stany Member Posts: 3

    Thank you Mack. It really helped.

  • TomF
    TomF Member, Employee Posts: 406 Solace Employee

    @Leo stany Assuming your .txt is small, this works well. What if your file is large? Like gigabytes? Well, another way to view this is the "event notification" pattern. Here you publish an event that details where the file is, its name etc. The consuming application uses this as a notification that the file is ready and that it should read it, getting the location of the file, size, who placed the file and other meta data from the event. It can then locate the file and do what it needs to do to read it.

  • Leo stany
    Leo stany Member Posts: 3
    edited March 2021 #6

    @TomF Thank you. But it is just a notification and there won't be any secure way of consuming a large file as a message. Could
    you tell me how to locate the file and read the file using solace queues after receiving the notification?

  • TomF
    TomF Member, Employee Posts: 406 Solace Employee

    You would need to put the location of the file in the message payload so that your receiving application could locate it. If you need to store the file in a queue, as you mentioned, then you'll need to follow @amackenzie 's advice above, but you'll be subject to the maximum message size of 30MB. If your file is bigger than that, you'll need to split the file in to chunks and re-assemble it.