Maximum number of Semp V2 calls at a time?

Manikanta
Manikanta Member Posts: 17 ✭
edited February 2021 in PubSub+ Event Broker #1

We have 3000+ queues in a appliance. We would like get all the queues information in JSON. We have a Python application Which makes SEMP V2 GET call to get all these queue information.
Is it okay to issue concurrent SEMP v2 GET calls to the appliance ?
I believe appliance will process the API call one after the other.
We will run this application once in a week to get the queue information.
@Aaron

Tagged:

Comments

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #2

    Yes, it is ok. But I would actually suggest that you strongly consider using SEMPv1 for this particular use case. As of the time of this writing, SEMPv1 performance still significantly exceeds SEMPv2, and if you're pulling back information for 3k queues, that could take some significant time in SEMPv2. (note, there are performance enhancements being made, just not sure when they are ready). SEMPv2 is great for configuration, where you have the full RESTful verbs/methods to build objects. But just fetching data, especially for a large number of queues/objects, I'd suggest (at least looking at) SEMPv1. Just means that you have to parse XML instead. But is much faster.

    Send a POST with the payload:

    <rpc>
  <show>
    <queue>
      <name>*</name>
      <vpn-name>*</vpn-name>
      <detail/>
      <count/>
      <num-elements>100</num-elements>
    </queue>
  </show>
</rpc>

    to the broker management port with /SEMP as the URL path. It will return 100 elements on one "page", and then at the end of the response will give you a "more cookie" for you to get the next page. Repeat until all done. Will be much faster than asking for 3,000 individual queues via SEMPv2.

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #3

    I should also probably answer the question asked though too... I believe the broker can have up to 500 simultaneous SEMP calls against it, but the recommendation from Solace is to limit to 10 calls per second. So you shouldn't ask for each queue individually, but in batches (similar to my SEMPv1 advice above).

  • Manikanta
    Manikanta Member Posts: 17 ✭
    #4
    1. What if we would like to get the subscriptions data of each queue, then we have to make those many calls right ? or Is there any work around for that in SEMPv2?
    2. Need to get the queue name first and then another hit for subscriptions for that queue right? Is there any way to get the subscriptions list as well along with queue details?
  • Manikanta
    Manikanta Member Posts: 17 ✭
    #5
    1. What if we issues more than 10 calls/sec or simultaneous SEMP v2 calls, is there any chance of appliance go down? or Will it effect the performance of the appliance? or Will the appliance give error response in back ? or Will it process all of them in anyway?
  • Manikanta
    Manikanta Member Posts: 17 ✭
    #6

    @Aaron Is there any similar call in SEMP v1 to get all subscriptions from all the queues? It is possible in cli, which gives all the information in one call.

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #7

    Hey @Manikanta sorry for the delay. Ok, let's respond to your questions:

    1. SEMPv2 has a way to get all the queue subscription info: https://docs.solace.com/API-Developer-Online-Ref-Documentation/swagger-ui/config/index.html#/queue/getMsgVpnQueueSubscriptions Or if you want to try using SEMPv1 like my example above, then just replace <detail/> with <subscriptions/>
    2. I'm not sure if you can use a wildcard with SEMPv2. Maybe someone else here knows?? But with older SEMPv1, you can use wildcards.
    3. The appliance will not go down, and should not impact messaging performance.. there is separation internally between the control plane and the data plane. However, if you are overdriving the SEMP interface, it might make other management functions slower (e.g. configuration activities, CLI access). You might start to get SEMP errors saying "cannot initiate a SEMP request" or something like that if you push too hard, a good indication to back-off your polling rates.
    4. For SEMPv1, see my #1 above. But yes, also possible in CLI if you want a text dump. Do no paging to turn off paging, and then do show queue * subs to dump out all queue and subscription details to the console. Can then cut-and-paste that into notepad or something.

    Hope that helps!

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #8

    I guess, to be crystal clear, if you wanted to use SEMPv1, you could (as an example, using curl):

    curl -u admin:admin http://localhost:8080/SEMP -d '<rpc><show><queue><name>*</name><subscriptions/><count/><num-elements>100</num-elements></queue></show></rpc>'

    NOTE 1: if appliance, you'll be using port 80 on the management plane, and not 8080
    NOTE 2: don't use admin credentials for monitoring!!!! Create a read-only user specifically for monitoring.
    NOTE 3: if you have more than 100 queues, you'll have a "more-cookie" at the end of the response... that is what you ask back to the appliance to get the next batch of results.

  • Manikanta
    Manikanta Member Posts: 17 ✭
    #9

    Thank you for your detailed response @Aaron

Categories

This Month's Leaders

This Week's Leaders