Good practice to differentiate business messages from technical messages

soutchilin

Hi,

If I need to validate data that are sent as payload in messages, I'd probably will use a validation microservice. It could be used as a in a request/response manner or as a filter that forwards messages through it. Anyway, if I get more and more on combinations of validation services and to be validated event types in my application domain, I have a mix of this technical messages and business events. So it will "blur" the view on the real business applications and event streams in the Event Poral Designer. Is that right? Is there a good approach to mitigate it?

I'm interested on that both from architectural and tooling points of view.

Cheers,
Alexander

Tamimi

Hey @soutchilin - if I understood your question correctly, you are interested to know more about architecture design as opposed to application implementation. This is an interesting question because I believe you can have multiple approaches to handle it. A first thought on this, I would look at topic hierarchy design. I would recommend giving this blog a read: Topic Hierarchy and Topic Architecture Best Practices, and check out the Topic Architecture Best Practices docs section on our website.

In a nutshell, you can have a rich topic hierarchy structure that will enable you to differentiate multiple business events directly in the topic. So for example, events from your Business Operation A could be published on topicLevel/level2/OpsB/level4 and events from your Business Operation B could be published on topicLevel/level2/OpsA/level4. That way, your validation microservice can validate the payload of the business operation it is interested in by just subscribing to the right topic using wildcards. You can also have several validation microservices each subscribing to the business operation of interest. This way, you have the ability to decouple your "technical messages" (in the payload of the event) and your "business events" (defined in the topic).

From an Event Portal Design perspective, you can avoid the "blur" you are referring to by having every business event as an application sending technical messages on different topics in your application domain.

I hope this answers your question, or at least this is what you were wondering about?

I would also see what others in the community have to say about this.

TomF

Hi @soutchilin,

This is an interesting question. @Tamimi is right in that good topic design is the essence of Solace's opinion on how this should be dealt with. Developing a good topic taxonomy (see the blog post introducing the idea ) will go a long way to helping.

I have seen people put a distinguishing level in their topic taxonomy for technical and business events. I personally don't like this, but I see the logic behind it and I think it is a valid thing to do. Please just ensure that it is part of the dynamic part of the topic, not the root. Ideally, as a consumer I'd like to be able to consume all events of a certain type, then filter out which are business events, rather than filter out all business events, then pick the event type.

I understand your concern about "clutter" in Event Portal. There are several ways you could deal with this. Having the events separated in the topic taxonomy will always help. You can also add a tag to events of a particular type, which can then be highlighted in Event Portal to make distinguising between the types easy. You could always separate the events into separate application domains.

Lastly, you mentioned using a payload validation service. Rather than using request/reply or some kind of synchronous "fronting" valiation service, I would say the best pattern is to emit the unverified events on a topic. Have your validation service subscribe to this topic, validate the payload and the emit the validated events back on to another topic. Use the event format part of the topic taxonomy to distinguish between the two. This gives you complete decoupling, enabling easier scaling of the validation service, addition of different types of validation in a staged and scalable way (Is the valid json? Yes. Is this valid json in the format my inventory system expects? Yes. Does this partID exist? No...) You can then chose how the validation service deals with failures - it could emit subsequent events, just add a log entry... the choice is yours.

Tamimi

Hey @soutchilin - if I understood your question correctly, you are interested to know more about architecture design as opposed to application implementation. This is an interesting question because I believe you can have multiple approaches to handle it. A first thought on this, I would look at topic hierarchy design. I would recommend giving this blog a read: Topic Hierarchy and Topic Architecture Best Practices, and check out the Topic Architecture Best Practices docs section on our website.

In a nutshell, you can have a rich topic hierarchy structure that will enable you to differentiate multiple business events directly in the topic. So for example, events from your Business Operation A could be published on topicLevel/level2/OpsB/level4 and events from your Business Operation B could be published on topicLevel/level2/OpsA/level4. That way, your validation microservice can validate the payload of the business operation it is interested in by just subscribing to the right topic using wildcards. You can also have several validation microservices each subscribing to the business operation of interest. This way, you have the ability to decouple your "technical messages" (in the payload of the event) and your "business events" (defined in the topic).

From an Event Portal Design perspective, you can avoid the "blur" you are referring to by having every business event as an application sending technical messages on different topics in your application domain.

I hope this answers your question, or at least this is what you were wondering about?

I would also see what others in the community have to say about this.

TomF

Hi @soutchilin,

This is an interesting question. @Tamimi is right in that good topic design is the essence of Solace's opinion on how this should be dealt with. Developing a good topic taxonomy (see the blog post introducing the idea ) will go a long way to helping.

I have seen people put a distinguishing level in their topic taxonomy for technical and business events. I personally don't like this, but I see the logic behind it and I think it is a valid thing to do. Please just ensure that it is part of the dynamic part of the topic, not the root. Ideally, as a consumer I'd like to be able to consume all events of a certain type, then filter out which are business events, rather than filter out all business events, then pick the event type.

I understand your concern about "clutter" in Event Portal. There are several ways you could deal with this. Having the events separated in the topic taxonomy will always help. You can also add a tag to events of a particular type, which can then be highlighted in Event Portal to make distinguising between the types easy. You could always separate the events into separate application domains.

Lastly, you mentioned using a payload validation service. Rather than using request/reply or some kind of synchronous "fronting" valiation service, I would say the best pattern is to emit the unverified events on a topic. Have your validation service subscribe to this topic, validate the payload and the emit the validated events back on to another topic. Use the event format part of the topic taxonomy to distinguish between the two. This gives you complete decoupling, enabling easier scaling of the validation service, addition of different types of validation in a staged and scalable way (Is the valid json? Yes. Is this valid json in the format my inventory system expects? Yes. Does this partID exist? No...) You can then chose how the validation service deals with failures - it could emit subsequent events, just add a log entry... the choice is yours.