[New blog post] How to generate dynamic topics from REST input headers

Version 9.11 of the Solace PubSub+ Event Broker introduced a great new feature for REST Delivery Points (RDP) to enable post request targets and HTTP request headers to be generated dynamically using substitution expressions.

This new feature can be used on egress to enable message specific variables, such as part of the topic address, to be included in the external HTTP request target. This can be extremely useful, for example to forward messages to storage services such as Azure Storage, Amazon S3, or Google Cloud Storage, where this requires sending each message to a unique target, preferably with that uniqueness coming from the message topic, metadata, or the current time.

For details on using substitution expressions in RDPs see here.

We can also make use of this new feature with REST messaging on ingress to generate dynamic topics from the HTTP input headers. When publishing messages we typically want to use a rich topic hierarchy to enable fine grained subscription management and routing, and we often want to include dynamic information from the message itself in the topic address.

Consider the example below. An order handling application is publishing new orders to a PubSub+ Event Broker using REST messaging and it's not possible for this application to generate the REST Request Target dynamically, only a static target can be configured. However, we want to route orders to different downstream fulfilment applications based on the type of order, for example one receiving application for order type ABC and a different receiving application for order type XYZ. How can we do this?

In this blog we'll show how to implement this example using an RDP to post the REST request back to the event broker using a substitution expression to generate a dynamic topic including the order type and code from the original message.

Login to PubSub+ Manager and create the following configuration:

1. Create a new queue, for example named "orders-rdp-q"
2. Create a subscription on the queue for the incoming orders topic: "input/orders"
3. Select the Clients page then REST tab and crate a new REST Delivery point, for example named "orders-rdp"
4. Select the "orders-rdp" REST Delivery Point and create a new REST Consumer, for example named "orders-rc"
5. In the REST Consumer settings set the Remote Host to "localhost" and port to the broker's REST messaging port, for example the default non-secure port is 9000. Optionally enable TLS and/or the relevant Authentication Scheme if required.

Your REST Consumer configuration should appear as follows:

7. Now create a Queue Binding for your RDP, and select the queue created earlier, for example "orders-rdp-q"
8. Configure the queue binding to use Substitution Expressions for the Request Target Evaluation and configure the Post Request Target with the following expression:

    /orders/${userPropertyAsString("Type")}/${userPropertyAsString("Code")}

Your Queue Binding configuration should appear as follows:

This will create a URI of the form orders/{type}/{code} where the Type and Code fields are populated dynamically from the values of these user properties in the message.

Your RDP should now show as operationally "Up", however if it shows as "Down" check for errors under the REST Consumer Stats tab and Details sub-tab.

To test the configuration, go to the "Try Me!" page and run a test subscriber to topic: "orders/>"

Now send test REST messages from a terminal using the curl command, for example:

    curl -X POST broker-host:9000/input/orders \
      -H "Solace-User-Property-Type: typeABC" \
      -H "Solace-User-Property-Code: code123" \
      -d "This is an order message"

    curl -X POST broker-host:9000/input/orders \
      -H "Solace-User-Property-Type: typeXYZ" \
      -H "Solace-User-Property-Code: code789" \
      -d "This is another order message"

Note that the header properties need to be prefixed with "Solace-User-Property-" as described here.

You should now see the test messages appear on the Try Me subscriber, note the topics have been dynamically generated to include the order type and code form the input headers of the original message.