Publish REST Consumer payload with x-www-form-urlencoded

theironcook

I'm investigating if I can possibly replace an old web hooks implementation with Solace (I sure hope so). This article has been a great read so far: https://solace.com/blog/how-to-maximize-microservices/

Sadly, my product has thousands of existing web hook subscriptions that rely on the the payloads being sent as x-www-form-urlencoded payloads. How can I create a REST Consumer in solace that will send a POST with the data x-www-form-urlencoded. Thanks,

marc

@theironcook it looks like you can actually stand up a v9.11 broker in Cloud today if you choose a GCP region :)

Aaron

Hi @theironcook ..! Welcome to the community. For my lack of understanding, can you elaborate a bit more? Do you need to change the payload encoding itself, or do you need to set the "content type" header to application/x-www-form-urlencoded and the rest stays the same?

If the former, Solace (the broker) doesn't do payload transformation... you'll need some intermediate microservice that does that... Spring Cloud Stream processor app or iPaaS or something. Not hard to do/write, but not something built into the broker.

If the latter (payload already right format), you just need to set the Content Type of the webhook going out, then that's not hard. In SolOS 9.11 which came out in Sept (2021), you can add additional headers to your outbound webhook for the RDP. Its actually part of the queue-binding configuration. Here are the CLI commands, change identifier names as appropriate.

enable
config
message-vpn default
rest
rest-delivery-point rdp1
queue-binding q/rdp1/1
create request-header content-type
header-vaue application/x-www-form-urlencoded

That's it! Now when you send a message in Solace that lands on the queue for your RDP (`q/rdp1/1`) in my case above, the message will have that additional HTTP header set to the backend REST service:

Received message: POST /OUTBOUND/ at 15:51:52
  HEAD:host = 172.19.4.248:9999
  HEAD:solace-aaron-custom =
  HEAD:cache-control = no-cache
  HEAD:content-length = 5
  HEAD:content-type = application/x-www-form-urlencoded
  HEAD:solace-delivery-mode = Persistent
  HEAD:user-agent = Solace_PubSub+_Standard/9.12.0.15
  BODY: hello

You can also set this in the GUI PubSub+Manager, under Clients->REST->Queue Bindings:

Hope that helps!

theironcook

Thank you Aaron. That's really helpful.

I can't see "Request Headers" as an option. I believe it's because my broker version is too old?

"In SolOS 9.11 which came out in Sept (2021), you can add additional headers to your outbound webhook for the RDP."

I'm using Solace Cloud on a free trial account. I don't see how I can upgrade my broker from "version 9.10.0.23". I also created a new account to create a brand new broker but the only options I can see for the cloud broker are "9.6 or 9.10 (default)".

I actually don't think this technique is going to work for me though. The more I dug into my use case, the more I realized I'm going to need to transform the queue payload, perform some custom encryption etc.

I'm most likely going to need a transformation step after processing the queue data like you mentioned. I might use something like a custom microservice or aws lambda. The web service endpoint will be synchronous. Basically I think I'll plan to have a new Solace REST Delivery Point that will invoke my new aws lambda function synchronously. The lambda function will perform the necessary payload transformations and invoke the original web hook endpoints. If the lambda function fails (or the downstream web hook fails), the Solace REST Delivery Point will fail and the message requeues in Solace. Does this sound correct to you?

Thanks,

Aaron

Hi again @theironcook. Yes, that sounds correct/fine to me. I've never played with Lambdas myself, but I know we have some resources on how to integrate:

• https://docs.solace.com/Configuring-and-Managing/HTTP-Header-Authentication-AWS-API-Gateway.htm
• https://solace.com/blog/event-broker-cloud-lambda-aws-api-gateway/
• https://docs.solace.com/Developer-Tools/Integration-Guides/Aws-Integration.htm

Note that if the downstream system from the RDP fails (the Lambda) b/c its downstream system fails, and it returns a 404 or 500 or whatever, the RDP will attempt to redeliver. Continuously. My colleague ran into an issue where it tried for days. So having something like a Max-Deliveries count on the queue, or setting a TTL, and having the message expire to a DMQ might be useful during testing.

Note that it might be possible/desirable for the Lambda to POST back into Solace, and from there another RDP can push it into your original webhook endpoint. But while the transformed message is going through Solace, other interested applications (e.g. analytics, real-time dashboards, etc.) could also snoop the messages because its just pub/sub, and you just need to subscribe to the same topic(s) as your 2nd RDP would (coming from the Lambda).

Too bad Solace Cloud doesn't have 9.11 yet. But I just saw an email recently from the Cloud team saying that they increasing their cadence for getting released Software brokers available in the Cloud, and also decreasing the time since the software is released. If you really want to try this option, just to kick the tires, it's really simple to get a local Docker broker running. https://solace.com/products/event-broker/software/getting-started/

Good luck! Keep us posted.

marc

@theironcook it looks like you can actually stand up a v9.11 broker in Cloud today if you choose a GCP region :)