PERFORMANCE! Why you should use Message.reset() in Solace APIs..!

Aaron
Aaron Member, Administrator, Moderator, Employee Posts: 497 admin
edited October 2022 in Tips and Tricks #1

Hi Developers! Want to go fast? Faster, anyway? Read on...

I'm just putting the finishing touches on a new bunch of Solace Java (JCSMP) samples, and have been doing a bit of performance testing with them. They are still meant to be fairly simple and straightforward, and don't try to maximize absolute throughput performance, but they're pretty quick. When publishing Direct messages in a loop, I was wondering what the difference would be between creating new Message objects each time, or reusing the same ones and calling reset() on each loop, and reusing it on the next loop. Well...

Creating new Message object each loop

Published msgs/s: 251,419
Published msgs/s: 255,510
Published msgs/s: 256,729
Published msgs/s: 258,469
Published msgs/s: 256,312

Using Message.reset() on each loop

Published msgs/s: 323,841
Published msgs/s: 316,270
Published msgs/s: 327,877
Published msgs/s: 336,641
Published msgs/s: 325,477
Published msgs/s: 326,037

Huh! About a 25% increase in speed. Not bad..! Also, when Java's GC would start to kick in, that would definitely periodically impact performance, having to clean up all these unused Message objects.

Technical notes: JCSMP API, single thread, Direct messages, 100 byte payload, dynamic topics, Intel(R) Core(TM) i7-7820X CPU @ 3.60GHz, 1 GbE LAN to Solace broker
Code is here: https://github.com/SolaceSamples/solace-samples-java-jcsmp/blob/master/src/main/java/com/solace/samples/jcsmp/patterns/DirectPublisher.java

Comments

  • Tamimi
    Tamimi Member, Administrator, Employee Posts: 489 admin

    I can picture the messages flowing through the broker like this

  • TomF
    TomF Member, Employee Posts: 405 Solace Employee

    Very useful bit of research, @Aaron. I wonder what the difference would be in the C API? Obviously not re-using messages in C would look a like a memory leak, which you would (hopefully) spot sooner or later, but all those extra mallocs would definitely slow things down.

  • Ben
    Ben Member, Employee Posts: 3 Solace Employee

    One precision our C API, it doesn't in most case use malloc at runtime due to performance issue, it preallocate buffers at startup and manage the pool. solClient_msg_alloc() and solClient_msg_free() are taking and returning buffers to the pool. Although even with buffer pool, it would have some impact on performance.

  • TomF
    TomF Member, Employee Posts: 405 Solace Employee

    @Ben of course, how could I forget _msg_alloc()! Time to write more code!

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 898 admin

    hey @Aaron I assume this was using a hardware appliance and not a sw broker? Is that correct?

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 497 admin

    Hey @marc , yup correct! Was an old 3260 8x1GbE we have in our Singapore lab.

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 497 admin
    edited October 2023 #8

    Old thread bump! I recently ran into a bug (i.e. I wrote bad code) where I was using message.reset() in a Guaranteed message publishing JCSMP app. Unlike with Direct messaging publishers, the broker sends ACKs and NACKs back to the publisher to indicate success for each Guaranteed message.

    A common pattern is to use the published message object itself as the correlation key. This way, if the publisher receives a NACK, it simply grabs the key as the message, and then requeues it to try to resend. And here is my problem!

    I was calling reset() on my single message object and reusing it each loop (like in my Direct messaging sample above). This was fine until I received a NACK and tried to get my message from the correlation key. But I had overwritten the message contents with following messages. Bad news!

    So either:

    a) create a new message object each loop for Guaranteed messages

    b) don't use the message object as the correlation key… keep all the payloads, topics, metadata in some data structure and use some other identifier as the key, and if you receive a NACK: rebuild the message from that source data.

    Note that for Direct message publishing, reset() is totally fine as the broker will never ACK/NACK your message. It's fire-and-forget. Oh, you may get a publish error due to publish ACL or something, but the correlation key will be null.

    Hope this helps someone out there..!

  • Ben
    Ben Member, Employee Posts: 3 Solace Employee

    And what about having a pool of messages for Guaranteed Messaging, that go back to free pool when message.reset() ? Then when there is no free message in the pool, you either stop publish or extend the pool ?