Weird chars at start of text payload!?

Hi all! I'm making a definitive post about this because it's been asked countless times, and I still can't find a good "one-page" reference response. The issue: sometimes you'll see weird characters at the beginning of your text payload. For example, this is from the JCSMP HelloWorld sample:

Destination:                            Topic 'solace/samples/jcsmp/hello/aaron'

Priority:                               4

Class Of Service:                       USER_COS_1

DeliveryMode:                           DIRECT

Message Id:                             6

Binary Attachment:                      len=26

1c 1a 48 65 6c 6c 6f 20    57 6f 72 6c 64 20 66 72    ..Hello.World.fr

6f 6d 20 41 61 72 6f 6e    21 00                      om.Aaron!.

Or if you just print out the raw payload as a String: ∟↓Hello World from Aaron!

See those first two bytes? 0x1c and 0x1a? What are they? In Solace world, when sending a TextMessage, you are not just sending a raw String as binary payload. The API is actually constructing a Structured Data Type (SDT) formatted message, where there is a single field (the String) in the container. Those first few bytes (could be 2-6 I think) define the size of the String contained within the SDT TextMessage.

Solace messages can be one of: TextMessage, BytesMessage, MapMessage, or StreamMessage. Thanks to JMS for these.

So your receiver/consumer should ideally check what type of message it is receiving and deal with it appropriately. In JCSMP Java, this looks something like:

public void onReceive(BytesXMLMessage message) {
    if (message instanceof TextMessage) {
        TextMessage msg = (TextMessage)message;
        String payload = msg.getText();
        // do more
    } else if (message instanceof BytesMessage) {
        BytesMessage msg = (BytesMessage)message;
        byte[] payload = msg.getData();   // NOT getBytes() strangely, that gets the XML payload
        // are you sure the payload is a string?
        String strPayload = new String(payload, StandardCharsets.UTF_8);
        // more
    } else if (message instanceof MapMessage) {  // not often used anymore
        MapMessage msg = (MapMessage)message;
        SDTMap map = msg.getMap();
    } else if (message instanceof StreamMessage) {  // not often used anymore
        StreamMessage msg = (StreamMessage)message;
        SDTStream stream = msg.getStream();
    } else {
        // should be impossible, these are the only 4 types
    }
    . . .

The newer Java API hides this stuff from you, BTW. Other APIs, I'm not sure..?

But in JavaScript/NodeJS, same thing… you need to check what type of message you've received and deal with appropriately:

session.on(solace.SessionEventCode.MESSAGE, function (message) {
    if (message.getType() == solace.MessageType.TEXT) {
        var strPayload = message.getSdtContainer().getValue();
        // do stuff
    } else if (message.getType() == solace.MessageType.BINARY) {
        var payload = message.getBinaryAttachment(); // binary attachment, could be String or Uint8Array
        // do stuff
    } else {
        // either a stream or a map SDT
    }
    . . .

See JavaScript docs on MessageType, and on getType().

A lot of people stumble into this with JavaScript since getBinaryAttachment() returns (usually) a String. And might not notice if their publisher app (also probably JavaScript) is sending plain Strings as raw binary / BytesMessage, instead of an SDT TextMessage. This issue usually shows up when you start mixing different types of publisher languages, APIs, or protocols, and the apps are not all formatting messages the exact same way. (e.g. Java publisher TextMessages, JavaScript consumer).

Oh, and if you want to send text messages with JavaScript, do something like this:

var msg = solace.SolclientFactory.createMessage();
msg.setDestination(solace.SolclientFactory.createTopicDestination("hello/world"));
msg.setSdtContainer(solace.SDTField.create(solace.SDTFieldType.STRING, "here is my text."));

Note that this publishing at structured text vs. binary also applies to REST/HTTP Messaging publishers. This is binary:

curl -u user:pw http://localhost:9000/hello/world -d 'hello bytes message'

And this is a structured TextMessage:

curl -u user:pw http://localhost:9000/hello/world -H 'content-type:text/plain' -d 'hello text message'

See Solace REST encoding docs here on HTTP Content-Type Mapping to Solace Message Types.

Finally, the broker has some smarts built into it for helping with protocol translation. For example, if subscribing with an MQTT client on topic hello/world, and I publish the two (SMF) HTTP messages above (or equivalently, one binary message and one text message), then the MQTT consumer receives both string payloads correctly (no weird extra chars):

I really hope this post helps people, and I've included enough keywords for Google to pagerank it highly..! 😁