Pubsubplus-connector-spark and OAuth2

Connectors & Integrations
1

Hello Community,

we are struggling with the pubsubplus-connector-spark connector via Python to get authorised via OAuth2 with client_secret and client_id only without TLS on Databricks.
We can not get a valid token while connecting with the connector. The message is invalid_client and if I dive deeper the message is No Truststore Password Configured.
With a workaround it works: we are requesting the token and store it on Databricks in a file and reading it when connection via python and the connector.
But this is not what we want. We want to connect to AEM via OAuth2 with client credentials with id and secret only, without any certificates.
Is this possible or did I oversea something?

this is how we configured the connector:

struct_stream = spark.readStream.format("solace") \
    .option("host", host) \
    .option("vpn", vpn) \
    .option("solace.apiProperties.AUTHENTICATION_SCHEME",
    "AUTHENTICATION_SCHEME_OAUTH2") \
    .option("solace.oauth.client.auth-server-url",
    "https://my-AEM-server.hana.ondemand.com/oauth/token?grant_type=client_credentials") \
    .option("solace.oauth.client.client-id",
    "my-id!1234") \
    .option("solace.oauth.client.credentials.client-secret",
    "my-secrects=") \
    .option("solace.oauth.client.access-token","/dbfs/FileStore/*/*/token_file.txt") \
    .option("solace.oauth.client.token.refresh.interval", 110) \
    .option("solace.oauth.client.token.fetch.timeout", 3600) \
    .option("solace.oauth.client.auth-server.ssl.validate-certificate", False) \
    .option("queue", queue_name) \
    .option("connectRetries", 2) \
    .option("reconnectRetries", 2) \
    .option("batchSize", 1) \
    .option("partitions", 1) \
    .load()

Any help I would appreciate!

Many thanks in advance and help!

BR, Jens

2

Hi there, I’m Solly AI, I generate responses to your questions using the power of the Solace Agent Mesh:

Please remember that AI technology can make mistakes so ensure that you read our Disclaimer.

AI Generated responses are a work in progress, your help in evaluating my response would be awesome! If my response makes sense, please mark it as a solution or give a :+1:. If my response doesn’t make sense for your topic, please react to it with a :shaking_face:.

Looking at the documentation and community discussions, I cannot provide a complete answer about configuring OAuth2 authentication with client credentials for the Solace Spark connector without TLS, as the specific configuration details are not explicitly covered in the provided facts.

However, there are a few important points to note:

  1. OAuth authentication requires TLS to be configured on the event broker, as mentioned in the documentation: “The event broker must be configured to use TLS by setting a server certificate.”

  2. For OAuth2 with client credentials flow:

  • The OAuth role should be set to “resource-server” on the broker side
  • The client needs to provide an access token
  • The broker needs to be configured with matching client_id and client_secret

Given these requirements, it appears that:

  1. You cannot use OAuth2 authentication without TLS
  2. You would need to configure TLS on both the broker and client side

For the proper configuration steps and requirements, please refer to:

I recommend reviewing the TLS configuration requirements in the Solace documentation before proceeding with OAuth2 authentication setup.

3

Hello Solly-AI,

to my knowledge, you can can use OAuth2 and ignore the SSL certificate as described in the respective documentation with .option("solace.oauth.client.auth-server.ssl.validate-certificate", False).
Therefore, the question came up if we can use OAuth2 only with client-id and client-secrect without X.509: cert truststore, private key etc pp?
With the ‘normal’ Solace Python SDK I can do that but with the pubsubplus-connector-spark connector via Python it fails with the error message that a truststore is mandatory. Is there a workaround for this connector or do I/we did some work in the .option section of the configuration in python?

Many thanks to all!
BR, Jens

4

Hello community.

As we are still struggling with this topic, has anybody any idea to help me on this?

Kind regards and many thanks!
Jens