Integrating Solace PubSub+ with Logstash

arih
arih Member, Employee Posts: 125 Solace Employee
in Connectors & Integrations #1

Logging is a crucial part of your applications, especially enterprise applications. If you're still writing logs to a database, you really need to read this :) This post shows how to use Solace as the pipeline to your ELK stack, replacing your slow and heavy JDBC insert operations with a JMS send operations. Let Solace be the shock-absorber and decouple your applications from the logging systems, no back pressure FTW!
I'm writing a couple of other posts around logging, but here's the first one. What do you think? Still want to write to database and get that extra elapsed time waiting for JDBC inserts? Or risking the application to stuck only waiting for log-written confirmation? Or maybe you'd prefer Fluentd? :) I'm curios to hear your thoughts!

https://solace.com/blog/integrating-solace-with-logstash/

Tagged:

Comments

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #2

    Hi @arih,
    Thanks for sharing. In your blog you used Elastic stack running on GCP. Is there anything specific to GCP that you did or would your blog work with open source elastic stack as well?

    -Marc

  • arih
    arih Member, Employee Posts: 125 Solace Employee
    #3

    hey @marc, should work with any Logstash installation, just sometimes different version of Logstash might have a different file name or location for the config. I used Bitnami just as a quick pick on the marketplace, no specific reasons :)

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #4

    Awesome, thanks @arih. Looking forward to your next blog :)

  • chmielej
    chmielej Member Posts: 4 ✭
    #5

    Hi @arih,
    In your blog you were using JNDI based connection to Solace using basic auth. I wonder if you had a chance to test as well client certificate authentication?
    Best regards,
    Jacek

  • arih
    arih Member, Employee Posts: 125 Solace Employee
    #6

    hi Jacek,

    Unfortunately I have not tested it with cert auth last time. Did you try and find error?

    I think it's more on how we set it up on logstash JMS input side though, the Solace auth config is documented here.

  • chmielej
    chmielej Member Posts: 4 ✭
    #7

    Finally I was able to sort it out. This is the Logstash configuration that works with client certificate auth which should be added inside jndi_context section:
    'javax.net.ssl.keyStore' => 'pathToKeystore.jks'
    'javax.net.ssl.keyStorePassword' => 'keyStorePassword'
    'Solace_JMS_Authentication_Scheme' => 'AUTHENTICATION_SCHEME_CLIENT_CERTIFICATE'

  • arih
    arih Member, Employee Posts: 125 Solace Employee
    #8

    Glad that you worked that out, and much appreciated to share that with everybody :smiley:

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #9

    Thanks for sharing the solution @chmielej!

    Do you mind sharing your entire jms input config so it's clear for future folks that come across this thread?
    That would help answer a question such as do these options go under just input.jms or under input.jms.jndicontext

    Thanks again for sharing!

  • chmielej
    chmielej Member Posts: 4 ✭
    #10

    Here is my Logstash input config:
    jms {
    include_header => true
    include_properties => true
    include_body => true
    use_jms_timestamp => false
    destination => 'SolaceQueue'
    pub_sub => false
    jndi_name => '/jms/cf/default'
    jndi_context => {
    'java.naming.factory.initial' => 'com.solacesystems.jndi.SolJNDIInitialContextFactory'
    'java.naming.security.principal' => '@vpn'
    'java.naming.provider.url' => 'url'
    'javax.net.ssl.trustStore' => 'pathToTrustStore.jks'
    'javax.net.ssl.trustStorePassword' => 'trustStorePassword'
    'javax.net.ssl.keyStore' => 'pathToKeyStore'
    'javax.net.ssl.keyStorePassword' => 'keyStorePassword'
    'Solace_JMS_Authentication_Scheme' => 'AUTHENTICATION_SCHEME_CLIENT_CERTIFICATE'
    }
    require_jars => ['pathToRequiredJars']
    }

Categories

This Month's Leaders

This Week's Leaders