How to publish and consume an attachment(.txt) file along with each message in queue?

Leo stany

Is there a way to publish a message and an attachment(.txt) file to a queue? Can this message and the attached file be consumed using java interface?

amackenzie

Hi Leo, thanks for the question.
It really comes down to what you mean by "attachment". In PubSub+ you can have messages published that have a binary attachment. The contents of a binary attachment are up to you, so it certainly could be a serialized .txt file. The consuming applications would need to be aware of the format of that binary attachment in the message payload in order to know to deserialize to a .txt file. This can be done in a number of ways. A common way is publishing to a certain topic (which could then be subscribed to by the queue for persistence) and have your consumer applications consume from that queue. The message headers could be interrogated to see what topic the messages were published to and confirm that you should treat them as .txt in your consumer logic.
Some links to concepts I mention are:
https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Adding-Data-Payloads.htm
https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Publishing-Guaranteed-Messages.htm
https://docs.solace.com/PubSub-Basics/Core-Concepts-Endpoints-Queues.htm#topic-queue-mapping

amackenzie

Hi Leo, thanks for the question.
It really comes down to what you mean by "attachment". In PubSub+ you can have messages published that have a binary attachment. The contents of a binary attachment are up to you, so it certainly could be a serialized .txt file. The consuming applications would need to be aware of the format of that binary attachment in the message payload in order to know to deserialize to a .txt file. This can be done in a number of ways. A common way is publishing to a certain topic (which could then be subscribed to by the queue for persistence) and have your consumer applications consume from that queue. The message headers could be interrogated to see what topic the messages were published to and confirm that you should treat them as .txt in your consumer logic.
Some links to concepts I mention are:
https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Adding-Data-Payloads.htm
https://docs.solace.com/Solace-PubSub-Messaging-APIs/API-Developer-Guide/Publishing-Guaranteed-Messages.htm
https://docs.solace.com/PubSub-Basics/Core-Concepts-Endpoints-Queues.htm#topic-queue-mapping

Leo stany

Thank you Mack. It really helped.

TomF

@Leo stany Assuming your .txt is small, this works well. What if your file is large? Like gigabytes? Well, another way to view this is the "event notification" pattern. Here you publish an event that details where the file is, its name etc. The consuming application uses this as a notification that the file is ready and that it should read it, getting the location of the file, size, who placed the file and other meta data from the event. It can then locate the file and do what it needs to do to read it.

Leo stany

@TomF Thank you. But it is just a notification and there won't be any secure way of consuming a large file as a message. Could
you tell me how to locate the file and read the file using solace queues after receiving the notification?

TomF

You would need to put the location of the file in the message payload so that your receiving application could locate it. If you need to store the file in a queue, as you mentioned, then you'll need to follow @amackenzie 's advice above, but you'll be subject to the maximum message size of 30MB. If your file is bigger than that, you'll need to split the file in to chunks and re-assemble it.