How to retrieve the System logs using cURL command?

techrahul95
techrahul95 Member Posts: 20 ✭✭
in PubSub+ Event Broker #1

Hi Community,

Is there any option available to retrieve the System logs using cURL command in the shell script?

Kindly let me know if there is any option available.

Appreciate your support.

Thanks,

Rahul.

Best Answers

  • uherbst
    uherbst Member, Employee Posts: 129 Solace Employee
    #2 Answer ✓

    Ha! Once in my lifetime, I can add information on an answer from @Aaron :-)

    It is possible to retrieve logs via SEMP.

    Something like:

    curl -u admin:admin -d '<rpc><show><log><event></event></log></show></rpc>' https://mr-abcd:943/SEMP


    1. You have to use the management port (and protocol) to your broker.
    2. You have to use a valid user (can even be read only)
    3. There are only the newest 1000 log lines available

    The restriction to "1000 log lines available" is so important, that I fully agree with Aaron to use an external syslog server, if possible.

    Uli

    P.S.: If you ever wanted to know, where to get the XML-SEMP commands for SEMPv1:

    In the solace container, there is a tool "cli-to-semp".

    Just try it with

    cli-to-semp "show client-profile *"

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #3 Answer ✓

    Haha @uherbst..! Yup, you got the green tick mark! Nice. Ok, now let me add a bit on your answer..! 😉

    For completeness @techrahul95, and if you need them: here are some of my Solace Syslog config files. They're not perfect, but they're pretty close. https://github.com/aaron-613/solace-logging-config

    Using SEMPv1 to pull the logs with curl is good, but not ideal. Especially if you have a massive log file. Now, you CAN actually specify the number of lines you want... not limited to the last 1000. This will return the last 2000 lines. (not sure how many you can specify before things break haha) (SEMPv1 has a maximum return payload size of 32MB)

    <rpc>
  <show>
    <log>
      <event>
        <lines/>
        <num-lines>2000</num-lines>
      </event>
    </log>
  </show>
</rpc>


    Alternatively, if you don't want to use an external logging server with Syslog, you could copy the event log file off the broker. But you'll need to make a special file-transfer type username:

    solace102> en
solace102# con
solace102(configure)# create username ft password pw file-transfer
solace102(configure/username)# exit
solace102(configure)#
solace102(configure)# show username *

Configured Users                 User Type     Global       Default VPN
                                               Access Level Access Level
-------------------------------- ------------- ------------ -------------
admin                            cli           admin        none
ft                               file-transfer

    This will allow you to copy files off the shell, off the control plane. From your Linux shell, or using your favourite SFTP program:

    scp -P 2222 ft@localhost:/logs/event.log .

    Note the port 2222 for SSH access on the software broker. Then the file-transfer type users have their file root as /usr/sw/jail/ so that's why the path is /logs/ and not the full path.

    Anyhow! There's some extra info and ways of grabbing the event log (or system or command logs).

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    edited February 2023 #4 Answer ✓

    Hi @techrahul95... I think you misunderstood me, or I'm misunderstanding you. So just want to be clear:

    1. Using an external Syslog server you can have as many lines as you want. You could keep years' worth of logs.
    2. Using the SEMPv1 command above, you can ask for 10000 or 50000 lines if you want. Just be warned that you can't (probably) ask for a million or something. But I don't think there's a configured maximum. The "2000" above was just showing an example.

    Obviously the external Syslog server is the best option, as you don't have to query the broker any time you want to look at log files.

    Glad to have helped..!

Answers

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #5

    Hi @techrahul95, can I ask what form factor your broker is? Software, hardware, Docker, Cloud, etc..? Typically you would login to the shell/control plane of the broker and either view or copy the log files directly from there; they are located in folder /usr/sw/jail/logs/

    Another option is to setup an external Syslog server, and point your PubSub+ broker to that. I have some config files for rsyslog and syslog-ng if you'd be interested in that. Can even use a free tier AWS EC2 instance for a "free" logging server.

  • uherbst
    uherbst Member, Employee Posts: 129 Solace Employee
    #6 Answer ✓

    Ha! Once in my lifetime, I can add information on an answer from @Aaron :-)

    It is possible to retrieve logs via SEMP.

    Something like:

    curl -u admin:admin -d '<rpc><show><log><event></event></log></show></rpc>' https://mr-abcd:943/SEMP


    1. You have to use the management port (and protocol) to your broker.
    2. You have to use a valid user (can even be read only)
    3. There are only the newest 1000 log lines available

    The restriction to "1000 log lines available" is so important, that I fully agree with Aaron to use an external syslog server, if possible.

    Uli

    P.S.: If you ever wanted to know, where to get the XML-SEMP commands for SEMPv1:

    In the solace container, there is a tool "cli-to-semp".

    Just try it with

    cli-to-semp "show client-profile *"

  • techrahul95
    techrahul95 Member Posts: 20 ✭✭
    edited January 2023 #7

    Hi @Aaron and @uherbst ,

    I am agree with @uherbst, This is the solution I was looking for.

    I think exporting the system logs from external syslog server is not that tedious task. I was just exploring the options if available.

    Thank you for the answering. Appreciated your help.

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    #8 Answer ✓

    Haha @uherbst..! Yup, you got the green tick mark! Nice. Ok, now let me add a bit on your answer..! 😉

    For completeness @techrahul95, and if you need them: here are some of my Solace Syslog config files. They're not perfect, but they're pretty close. https://github.com/aaron-613/solace-logging-config

    Using SEMPv1 to pull the logs with curl is good, but not ideal. Especially if you have a massive log file. Now, you CAN actually specify the number of lines you want... not limited to the last 1000. This will return the last 2000 lines. (not sure how many you can specify before things break haha) (SEMPv1 has a maximum return payload size of 32MB)

    <rpc>
  <show>
    <log>
      <event>
        <lines/>
        <num-lines>2000</num-lines>
      </event>
    </log>
  </show>
</rpc>


    Alternatively, if you don't want to use an external logging server with Syslog, you could copy the event log file off the broker. But you'll need to make a special file-transfer type username:

    solace102> en
solace102# con
solace102(configure)# create username ft password pw file-transfer
solace102(configure/username)# exit
solace102(configure)#
solace102(configure)# show username *

Configured Users                 User Type     Global       Default VPN
                                               Access Level Access Level
-------------------------------- ------------- ------------ -------------
admin                            cli           admin        none
ft                               file-transfer

    This will allow you to copy files off the shell, off the control plane. From your Linux shell, or using your favourite SFTP program:

    scp -P 2222 ft@localhost:/logs/event.log .

    Note the port 2222 for SSH access on the software broker. Then the file-transfer type users have their file root as /usr/sw/jail/ so that's why the path is /logs/ and not the full path.

    Anyhow! There's some extra info and ways of grabbing the event log (or system or command logs).

  • techrahul95
    techrahul95 Member Posts: 20 ✭✭
    #9

    Thanks @Aaron. I have checked the repository. it is awesome 😃.

    I have understood that we can get maximum of 2000 lines of system logs but as for my use case it is not ideal. I will definitely check it out with SEMP v1 command.

    Appreciated your support.

    Regards,

    Rahul

  • Aaron
    Aaron Member, Administrator, Moderator, Employee Posts: 636 admin
    edited February 2023 #10 Answer ✓

    Hi @techrahul95... I think you misunderstood me, or I'm misunderstanding you. So just want to be clear:

    1. Using an external Syslog server you can have as many lines as you want. You could keep years' worth of logs.
    2. Using the SEMPv1 command above, you can ask for 10000 or 50000 lines if you want. Just be warned that you can't (probably) ask for a million or something. But I don't think there's a configured maximum. The "2000" above was just showing an example.

    Obviously the external Syslog server is the best option, as you don't have to query the broker any time you want to look at log files.

    Glad to have helped..!

  • techrahul95
    techrahul95 Member Posts: 20 ✭✭
    #11

    Thanks for the detailed insights @Aaron. I got your points clearly.

    Appreciated a lot. 🙂

Categories

This Month's Leaders

This Week's Leaders