C# library is allocation heavy on the Send() for message arrays, any chance for improvement?

Aleksei

I have market data app that uses 3rd party C++ library that I wrapped in native C interface and I'm using it in C# app. I exchange memory between native and managed with zero allocations and copies using all the latest and greatest stuff C# has to offer at .NET 6 level. But Solace C# client library is lighting up my application with allocations like a Christmas tree, and it is the only source of any allocations after initial startup of the app.

The allocation appear to come from two places in Send(IMessage[], int, int).

1. Dictionary for checking uniqueness of passed messages. It is allocated without consideration that message limit is 50 and it causes multiple reallocs due to growth... If the limit is known to be 50 and array is passed that has length attribute, why dictionary is at least not allocated with enough size initially? Room for improvement. Dictionary can be replaced by stackalloc array of 50 elements with basic linear search upon insertion of an id into it. This is only 49 passes of array which is nothing in terms of CPU compared to GC. ArrayPool<Int32> can alternatively be used to borrow memory on the heap if stackalloc use is not possible for some reason.
2. BitConverter.GetBytes(long) is used in both Send methods during call to TagMsgAtomically. There is a version of the method called TryWriteBytes(Span<byte>, long) since .NET Standard 2.1. It can be used with stackalloc or ArrayPool array to convert long into bytes without allocation.

Currently as a workaround I'm trying to find good moments in market stream to manually run GC cycles not to wait for garbage to pile up in 30 min interval long pause cleanups. Unfortunately the only garbage in my app is generated by C# client library and 3rd party C++ lib I use doesn't allow me to transparently see when a good moment for manual GC is due to some threading design decisions taken by it's authors and it being a black box. So improvements to the Solace C# client library are very welcome.

nicholasdgoodman

Very interesting observations - and absent any direct changes to the SDK library, I am looking to see if there are any other workarounds apart from your manual garbage collection scheduling.

Can you clarify, the performance issues you observe are not necessarily on the allocations themselves, but on the garbage collection that results from it? Or are both impacting the observed performance?

Aleksei

Hello @nicholasdgoodman,

In C# allocations are cheap, so my problem is payback on GC which wouldn't normally be needed if allocations didn't happen in the first place. I found a good place to induce GC manually, but I'm not happy with the result to be honest as time app spends in pauses is much longer in total time but pauses themselves are now of predictable lengths at least. This isn't really a workaround I would like long term.

Profiler also shows some additional allocations of temporary arrays like IntPtr[] and Int64[] during sends (array version, maybe single too) which can probably be avoided, but the ones I mentioned are multiple times more frequent and can easily be worked around with backward compatible changes to library code.

I'm a long time user of Solace product, must admit mostly in C++, but C# nowadays is becoming high performance language with all the memory management improvements of late. So I'm happy to sign an NDA just for a chance to contribute to C# client API or at least assist you in any ways without having access to source code.

nicholasdgoodman

I see. There are some internal discussions going on about this and, as you note, the optimizations are fairly straightforward apart from the fact a number of Solace users are still leveraging significantly older (perhaps even out-of-support) versions of .NET Framework. It would be possible to address this by changing what versions of .NET are supported or by using conditional compilation.

@Aleksei, that being said, while thinking about other workarounds and the general problem, you have got me curious about minimizing the impact of the GC, and I wanted to ask how are you handling the actual IMessage instances? Although you can manually control the unmanaged resources (alloc and free calls) via the provided .Dispose() implementation, are you avoiding GC on the managed instances somehow -- or are you re-using the messages and simply updating their payloads and headers?

Assuming there's no immediate fix to the package, are you interested in some (possibly hacky) workarounds to this issue? It may be possible to "rework" some of these methods using P/Invoke to grab some private IntPtr fields and DllImporting from the C API.

Even if not, if I can get a working example, I will share it (with the requisite caveats).

Aleksei

@nicholasdgoodman you are right about older frameworks, updating to .Net standard 2.1 if I remember correctly will drop the classic .NET framework out of the window. But stackalloc should be available in 2.0 and in the worse case if it isn't, it should be possible to use ConcurrentBag to store heap arrays of 50 elements for uniqueness validation in arrayed Send() replacing dictionary. I use ConcurrentBag to store protobuf object instances for market data in this app. I put a link at the bottom with example of making object pool with it.

With regards to IMessage instances, I Reset() them after Send accepts them on a wire and reuse them on next Send. Acknowledgements come with a correlation tag set to my internal objects, so I don't need IMessage instance beyond return of Send call.

Regarding the P/Invoke, this one is interesting, If I had pointers from the message instances and a session pointer I could call solClient_session_sendMultipleMsg directly and still use most of C# lib. Do you have an example or an idea if this is possible without modifying the library to expose those internal IntPtr? The last time I did trickery like that was a decade ago and it involved reflection.

https://learn.microsoft.com/en-us/dotnet/standard/collections/thread-safe/how-to-create-an-object-pool

nicholasdgoodman

So, I will start this with a disclaimer: the following is merely sample code which shows a hypothetical "how it could be done", and bypasses much of the internal sanity checks, validations, etc. that the full .NET SDK (C Wrapper) provides. Also, because it involves reflection to obtain private fields, could break any time you upgrade the .NET SDK package.

With that out of the way, here is a very bare-bones helper class which allows a "mostly C#" application to directly invoke the C send multiple API. Note: as we discussed, it assumes that individual IMessage instances are going to be re-used. (I would be curious to hear how you are handling ITopic references as well.)

class MessageBatchSender: IDisposable
{
    IntPtr sessionPtr;
    IntPtr[] messagePtrs;
        
    public MessageBatchSender(ISession session)
    {
        this.Messages = Enumerable.Repeat<object>(null, 50).Select(_ => session.CreateMessage()).ToArray();
        this.messagePtrs = this.Messages.Select(GetPrivateIntPtr).ToArray();

        this.Session = session;
        this.sessionPtr = this.GetPrivateIntPtr(session);
    }

    public IMessage[] Messages { get; }
    public ISession Session { get; }

    public ReturnCode SendMessages(uint length, out uint messagesSent)
    {
        return SolClientSessionSendMultipleMsg(this.sessionPtr, this.messagePtrs, length, out messagesSent);
    }

    // This method could stop working if an SDK upgrade changes the internal implementation
    private IntPtr GetPrivateIntPtr(IMessage message)
    {
        var messageImplType = message.GetType();
        var messagePtrField = messageImplType.GetField("m_opaqueMessagePtr", BindingFlags.NonPublic | BindingFlags.Instance);
        return (IntPtr)messagePtrField.GetValue(message);
    }

    // This method could stop working if an SDK upgrade changes the internal implementation
    private IntPtr GetPrivateIntPtr(ISession session)
    {
        var sessionImplType = session.GetType();
        var sessionPtrField = sessionImplType.GetField("m_opaque", BindingFlags.NonPublic | BindingFlags.Instance);
        return (IntPtr)sessionPtrField.GetValue(session);
    }

    public void Dispose()
    {
        //TODO: Dispose all those IMessage instances!
    }

    [DllImport("libsolclient", CharSet = CharSet.Ansi, EntryPoint = "solClient_session_sendMultipleMsg", ExactSpelling = true)]
    [SuppressUnmanagedCodeSecurity]
    static extern ReturnCode SolClientSessionSendMultipleMsg(IntPtr opaqueSession, [MarshalAs(UnmanagedType.LPArray)] IntPtr[] opaqueMessages, uint msgArrayLength, out uint numMsgsSent);
}

And it can be used in this manner:

// Create context and session instances
using (var context = ContextFactory.Instance.CreateContext(contextProperties, null))
using (var session = context.CreateSession(sessionProperties, null, null))
{
    // Connect to the Solace messaging router
    Console.WriteLine($"Connecting as {username}@{vpnname} on {host}...");
    var connectResult = session.Connect();

    if (connectResult == ReturnCode.SOLCLIENT_OK)
    {
        Console.WriteLine("Session successfully connected.");

        // Create a topic and subscribe to it
        using (var publisher = new Helpers.MessageBatchSender(session))
        using (var topic = ContextFactory.Instance.CreateTopic("tutorial/topic"))
        {
            // This example assumes all messages have the same topic
            publisher.Messages[0].Destination = topic;
            publisher.Messages[0].BinaryAttachment = Encoding.UTF8.GetBytes("Msg 1");
            publisher.Messages[1].Destination = topic;
            publisher.Messages[1].BinaryAttachment = Encoding.UTF8.GetBytes("Msg 2");
            publisher.Messages[2].Destination = topic;
            publisher.Messages[2].BinaryAttachment = Encoding.UTF8.GetBytes("Msg 3");
            publisher.Messages[3].Destination = topic;
            publisher.Messages[3].BinaryAttachment = Encoding.UTF8.GetBytes("Msg 4");

            Console.WriteLine("Publishing messages...");
            var sendResult = publisher.SendMessages(4, out var messagesSent);
                                                     
            if (sendResult == ReturnCode.SOLCLIENT_OK)
            {
                Console.WriteLine($"Done. Sent {messagesSent} messages.");
            }
            else
            {
                Console.WriteLine($"Publishing failed, return code: {sendResult}");
            }
        }
    }
    else
    {
        Console.WriteLine($"Error connecting, return code: {connectResult}");
    }
}