Using sdk_perf to generate 1 million unique topics

tkunnumpurath
tkunnumpurath Member, Employee Posts: 11 Solace Employee
in Tips and Tricks #1

I needed to do a test to generate messages with 1 million unique topics. Initially, I thought of writing a simple application that did this but then discovered that the free to download sdk_perf utility gives you this flexibility with the following parameters
-ptc=1000000 -ptp=topic/prefix/

This will generate incrementing topics starting with topic/prefix/000000 all the way to 999999. Note that if you are using java, you will have to increase the heap size significantly.

Tagged:

Comments

  • [Deleted User]
    [Deleted User] Posts: 25 ✭
    #2

    Hey great tip @tkunnumpurath! Thanks for sharing

  • hong
    hong Guest Posts: 480 ✭✭✭✭✭
    #3

    Thanks, @tkunnumpurath! This is really helpful.

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 957 admin
    #4

    @tkunnumpurath said:
    I needed to do a test to generate messages with 1 million unique topics. Initially, I thought of writing a simple application that did this but then discovered that the free to download sdk_perf utility gives you this flexibility with the following parameters
    -ptc=1000000 -ptp=topic/prefix/

    This will generate incrementing topics starting with topic/prefix/000000 all the way to 999999. Note that if you are using java, you will have to increase the heap size significantly.

    Nice, thanks @tkunnumpurath ! Also a hint for folks that want to give sdkperf a shot: you can of course see a bunch of options by running sdkperf with the -h help flag but you can also use -hm which will show you many more options the tool offers. I've found sdkperf super useful over the years but it has so many options that it's impossible (at least for me!) to remember them all.

  • himanshu
    himanshu Member, Employee Posts: 67 Solace Employee
    #5

    Ooh, I didn't know about the -hm option. That's extremely helpful. Thanks!

    @marc said:

    @tkunnumpurath said:
    I needed to do a test to generate messages with 1 million unique topics. Initially, I thought of writing a simple application that did this but then discovered that the free to download sdk_perf utility gives you this flexibility with the following parameters
    -ptc=1000000 -ptp=topic/prefix/

    This will generate incrementing topics starting with topic/prefix/000000 all the way to 999999. Note that if you are using java, you will have to increase the heap size significantly.

    Nice, thanks @tkunnumpurath ! Also a hint for folks that want to give sdkperf a shot: you can of course see a bunch of options by running sdkperf with the -h help flag but you can also use -hm which will show you many more options the tool offers. I've found sdkperf super useful over the years but it has so many options that it's impossible (at least for me!) to remember them all.

  • Ben
    Ben Member, Employee Posts: 3 Solace Employee
    #6

    I would add -he option from sdkperf...

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 957 admin
    #7

    @Ben said:
    I would add -he option from sdkperf...

    thanks @Ben. Somehow I never noticed those options before!

    Here is the output for those of you that are interested. It shows some example commands:

    MJD-MacBook-Pro.local:~/git$ sdkperf_java.sh -he
CLASSPATH: :./lib/commons-cli-1.2.jar:./lib/commons-lang-2.6.jar:./lib/commons-logging-1.1.3.jar:./lib/geronimo-jms_1.1_spec-1.1.1.jar:./lib/netty-buffer-4.0.41.Final.jar:./lib/netty-codec-4.0.41.Final.jar:./lib/netty-codec-http-4.0.41.Final.jar:./lib/netty-common-4.0.41.Final.jar:./lib/netty-handler-4.0.41.Final.jar:./lib/netty-transport-4.0.41.Final.jar:./lib/proton-j-0.14.0.jar:./lib/slf4j-api-1.7.21.jar:./lib/slf4j-simple-1.7.22.jar:./lib/sol-jcsmp-10.1.1.jar:./lib/sol-jms-10.1.1.jar:./lib/sol-sdkperf-8.0.0.12.jar:./lib/sol-sdkperf-jcsmp-8.0.0.12.jar:./lib/sol-sdkperf-jms-8.0.0.12.jar:./lib/thirdparty/*.jar:./lib/optional/log4j-1.2.17.jar:./lib/optional/
JAVA: /usr/bin/java
SOLACE_VM_ARGS: -Xms512m -Xmx1024m
  -------------------------------------------
  CLI Examples (for basic options see -h | -?)
  -------------------------------------------

  - Publish and receive 5 100 byte messages on a topic at 1 msg/sec:
      sdkperf_java -cip 192.168.160.xx -ptl a -stl a -msa 100 -mn 5 -mr 1

  - Provision a queue, map 4 topics to this queue and publish to the topics:
      sdkperf_java -cip 192.168.160.xx -pe -sql queue -stl a,b,c,d -ptl a,b,c,d -mn 100 -mr 100 -msa 100

  - Publish to queue with time to live of 1 millisecond and mark the message as DMQ eligible:
      sdkperf_java -cip 192.168.160.xx -pql queue -mt persistent -mn 100 -mr 100 -msa 100 -mtl 1 -mdq

  - Subscribe to topic 'a' with deliver always.  Useful in testing publish to one (-pto):
      sdkperf_java -cip 192.168.160.xx -stl "a<TOPIC_END/>DA=1"

  - Enter subscriptions on behalf of another client:
      sdkperf_java -cip 192.168.160.xx -stl a -scn c0001

  - Example of tuning API params with epl lists:
      sdkperf_java -cip 192.168.160.xx -stl a -epl "jcsmp.MESSAGE_CALLBACK_ON_REACTOR,true"

  - Connect a JCSMP client using SSL.  (For a complete list of SSL options see -hm)
      sdkperf_java -cip tcps://192.168.160.xx

  - Connect a JMS client using SSL.  (For a complete list of SSL options see -hm)
      sdkperf_jms -cip smfs://192.168.160.xx

Categories

This Month's Leaders

This Week's Leaders