The main problem is how to connect those many brokers to each other.
The best solution would be DMR or multi level DMR. Now multilevel DMR ist not available (not yet) and plain DMR has a connection limit of 15.
So as we currently face ~200 broker an growing.
DMR is not an option.
Those we choose an hub an spoke architecture , here all those micro broker are connect to an hub/concentrator and those concentrator to each other have a meshed bus.
Meshed to scale out and bus, because data is only allowed to flow from production to development stage and not vice versa.
Now you face the problem that you only want to transport messages that where you have a subscription on.
To solve this we invented a api management portal. Here you can publish an open api. This will allow the topics defined in the api, to flow in direction hub/concentrator.
If some one signs up this api the data will flow in his direction. There for the api portal have to configure all bridge queues automatically.
Because of sometimes complicated update pathes, we decided not to update broker. Meaning we have to create a new broker and delete the old one.
There for we created a self service portal to enable the dev teams to manage their own broker, on to of solace cloud, to be multi client (micro service owner) capable.
Next problem, you will be able to roll out your broker configuration like queues, clien and acl profiles, ….
There for we enable the teams to create zip artifacts containing json files describing those.
Allowing those json containing variables for von name, stage ….
This functional to allow to release a broker configuration tied to a micro service release, fast and easy.
We add this functionality to out self service portal.
This was allot of work, to read out current configuration and apply the div to the to state.
And allot of other small issues