🎄 Happy Holidays! 🥳

Most of Solace is closed December 24–January 1 so our employees can spend time with their families. We will re-open Thursday, January 2, 2024. Please expect slower response times during this period and open a support ticket for anything needing immediate assistance.

Happy Holidays!

Please note: most of Solace is closed December 25–January 2, and will re-open Tuesday, January 3, 2023.

Clarification on Multiprocessing Support in Solace PubSub+ Python API

hobart
hobart Member Posts: 3 ✭
in Early Access API #1

Hi,

I noticed on the Solace PubSub+ Python API page on PyPI that it mentions the messaging API does not support multiprocessing. Is this information outdated or still valid?

https://pypi.org/project/solace-pubsubplus/

Additionally, if we connect to a non-exclusive queue, having multiple clients using the Solace Python library seems like a very natural setup. Can you provide some clarity on this?

Thank you!

Tagged:

Answers

  • Ragnar
    Ragnar Member, Employee Posts: 67 Solace Employee
    #2

    There are no issues with multiple separate clients using the Solace Python Library whether on the same or separate platforms. So if you start multiple separate clients each with their own messaging service you are fine.

    What follows is some history and some rationale for the docs.

    There is an ancient concept, that predates multi-threading, of parallelism/concurrency through multi-processing. In this model the application forks (clones itself) into two processes that share only file descriptors, such as sockets for read/write. It introduces huge complexity into the application where it is no longer possible to share state information as each process has it's own memory space. Shared memory was invented to deal with this.

    But multi-threaded puts this all back in the OS and is what everyone is familiar with today.

    None of the Solace APIS support this type of multiprocessing where the session is shared between separate processes. It never even occurs to use it or warn against it because of multi-threading support.

    However Python, with it's Global Interpreter Lock, is very bad at multi-threading and consequently some developers have reverted to the ancient practice of multiprocessing, there is even a multiprocessing package that can be imported, and articles written about it such as

    Speed Up Your Python Program With Concurrency – Real Python

    This will not work, or will not work with the Solace Python API, you cannot access the same instance of the API from multiple different processes spawned by the multiprocessing package.

Categories

This Month's Leaders

This Week's Leaders