spring cloud stream multi api version

GreenRover
GreenRover Member Posts: 22 ✭✭
in Connectors & Integrations #1

The problem:
A single spring cloud stream application,
have to send out the same data via 2 major versions of the same api.

spring:
    stream:
      bindings: 
        myApiV1-out-0:
          destination: api/v1/the/topic
        myApiV2-out-0:
          destination: api/v2/the/topic

Question:
What is the most elegant solution for this problem?

Idea / what i want to:
My application should send to a generic binding using the stream bridge.
This binding should consume my internal model and

  • convert it to api mode V1 and send out via myApiV1-out-0
  • convert it to api mode V2 and send out via myApiV2-out-0
    Is this what i have to implement by my own or is there some kind of pattern?

https://stackoverflow.com/questions/69372670/spring-cloud-stream-multi-api-version

Tagged:

Comments

  • JamilAhmed
    JamilAhmed Member, Employee Posts: 20 Solace Employee
    #2

    Hi @GreenRover, can I suggest a different approach to consider:
    1. The application is always at whatever is the most current version. (v2 in this scenario.) It is responsible for just one event output at that version, always moving forward without any legacy burden.
    2. Have another service responsible for any necessary 'step down'. In other words, it just simply consumes the v2 that was output, has the logic to covert to the v1, and publish that event for every v2 event it sees.

    This would keep your main application free from continually handling this backward compatibility. Like it only gets worse as you move to a v3 soon right?
    The version step-down functions can then retire as support for the earlier versions retire too.

  • GreenRover
    GreenRover Member Posts: 22 ✭✭
    #3

    This wont work if:

    v2: contains a mandatory field (depreacted)
    v3: contains field not more

    If my app only publish latests (v3)
    The converter app is unable to add the deprecated field.

    It is a fixed requirement to support deprecated apis for at least 6 month to let consumers enough time to migrate.

    Always publish the oldest supported version will also not work in case if V3 add a new field.
    Because the business force us to publish the new field asap not only in 6 month

  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #4

    Hi @GreenRover,

    Hope you're doing well! I like @JamilAhmed's idea of separating the logic of the current version vs. potentially 1+ legacy versions, but you have a great point with the potential for the current version not containing deprecated information.

    How does your app receive the data? Is it delivered via a Solace topic to where you could just have multiple microservices receiving it? One for each version?

    If the data isn't already arriving on a solace topic you could obviously publish it to a topic to allow for fan-out to happen outside of your app, or if you prefer to do it inside your app then my gut reaction is that this feels like a Spring Integration type challenge and you could either leverage an internal spring integration pubsub channel or even use a Spring Integration Flow as a function.

    I'll try to think more about this later and let you know if I think of a more elegant solution.

  • GreenRover
    GreenRover Member Posts: 22 ✭✭
    #5

    Hi @marc , that we have to publish multiple major version of an api is a common pattern.
    The sources are completely mixed from REST/WebBrowser to inbound topics.
    The receives are multiple microservices those that we have to provide both version parallel.
    The major version of the api is part of the topic, this helps us to route the fan out traffic.

    I prefer it app internal.
    Do you have an example for:

    • internal spring integration pub sub channel
    • spring integration flow as function
      ?
  • marc
    marc Member, Administrator, Moderator, Employee Posts: 959 admin
    #6

    Hi @GreenRover,

    After thinking about this more I think using internal channels or using a spring integration flow would probably just add complexity. I think I like the approach that Soby suggested in his stackoverflow response here. Since you prefer it to be app internal then whenever a new version is introduced that app will already have to be modified, tested and re-deployed so might as well keep it simple so any java developer could understand what's going on. This allows you to just use regular conditional logic to decide the order in which you'd like to do things or not do things in the case of failures and avoid sending things over extra internal channels or introducing a Spring Integration flow which can be hard for someone not familiar with SI to follow.

    Unfortunately I don't have any samples for those two, but it looks this blog and the docs here further discuss using Spring Integration w/ Spring Cloud Function/Stream.

    Hope you have a good weekend!

  • GreenRover
    GreenRover Member Posts: 22 ✭✭
    #7

    Thanks for your response.
    You are right, keeping it as simple as possible, might be the better solution than a fancy not intuitive solution

Categories

This Month's Leaders

This Week's Leaders