Help with setting up appropriate endpoints.
Hi,
I have a pretty standard use case, but am struggling to work out a basic architecture, perhaps you can help?
We have a producer which would publish prices at random on a topic. We have a number of consumers who all need to consume these prices. When these consumers start up they need to obtain the last published price and subscribe to future price updates. How would you achieve this?
Would having a durable Queue for each consumer, to allow the replaying of all messages, and then iterate over all messages and by picking the last we can get the last price? We could then subscribe to future updates? This doesn't sound efficient...
Further we will have consumers, using the Python API, who will want to fetch the last price off the message bus on demand - without subscribing to future updates. Again iterating through all the stored messages in a pre-created durable Queue sounds like it might do the trick, but will slow down during the day (as more messages are published).
Can we create a Queue of length 1?
Can we avoid having to setup a queue for each consumer?
Many thanks
A
Best Answers
-
Hi @AlexanderJHall,
Can we create a Queue of length 1?
Yes! The easy approach here is to use Solace's "Last Value Queue" (LVQ for short) feature. This is indeed a queue that only holds the last message received by a queue. So a single app could bind to the LVQ to get the latest price while it was offline and then receive price updates while it's online.
Can we avoid having to setup a queue for each consumer?
I think you may be able to leverage PS+ Cache for this but that isn't my area of expertise so I'll tag in a few folks that might know it better ;) @Aaron @TomF @himanshu @nram
0 -
Hi @AlexanderJHall ,
Are these Direct or Guaranteed Messages? If they are Direct, you can use PS+ Cache for this exact usecase where you consumers want to connect to broker, subscribe to topic, get last message and then receive future updates.
Solace PubSub+ Cache is a scalable, in-memory message cache for Direct messaging. It allows client applications connecting to Solace PubSub+ to request the most current messages for a topic of interest when they come online, or when they start subscribing to topics that they were not originally subscribed to. For example, a real-time stock quote display may need to retrieve the latest value for a symbol when a user expresses interest in monitoring that symbol. In such a situation, it is useful to have a standalone application that saves the last messages sent for a topic of interest, and responds to client requests for copies of the messages published to that topic.
PS+ Cache is a separate add-on component that you can purchase. More info about it here.
If you are dealing with Guaranteed Messages, you can leverage LVQ which is simply a regular queue with depth of 0. Such a queue will always hold the last value only. You will have to create an LVQ for each corresponding queue though.
2 -
@AlexanderJHall - Unfortunately, I don't believe the python API supports PS+ Cache yet. It is in the roadmap to do so at some point. I would recommend letting your Solace Account team know that you are interested i this and they can raise the request officially.
@amackenzie FYI
1
Answers
-
Hi @AlexanderJHall,
Can we create a Queue of length 1?
Yes! The easy approach here is to use Solace's "Last Value Queue" (LVQ for short) feature. This is indeed a queue that only holds the last message received by a queue. So a single app could bind to the LVQ to get the latest price while it was offline and then receive price updates while it's online.
Can we avoid having to setup a queue for each consumer?
I think you may be able to leverage PS+ Cache for this but that isn't my area of expertise so I'll tag in a few folks that might know it better ;) @Aaron @TomF @himanshu @nram
0 -
Hi @AlexanderJHall ,
Are these Direct or Guaranteed Messages? If they are Direct, you can use PS+ Cache for this exact usecase where you consumers want to connect to broker, subscribe to topic, get last message and then receive future updates.
Solace PubSub+ Cache is a scalable, in-memory message cache for Direct messaging. It allows client applications connecting to Solace PubSub+ to request the most current messages for a topic of interest when they come online, or when they start subscribing to topics that they were not originally subscribed to. For example, a real-time stock quote display may need to retrieve the latest value for a symbol when a user expresses interest in monitoring that symbol. In such a situation, it is useful to have a standalone application that saves the last messages sent for a topic of interest, and responds to client requests for copies of the messages published to that topic.
PS+ Cache is a separate add-on component that you can purchase. More info about it here.
If you are dealing with Guaranteed Messages, you can leverage LVQ which is simply a regular queue with depth of 0. Such a queue will always hold the last value only. You will have to create an LVQ for each corresponding queue though.
2 -
Thanks guys, that's really helpful!
Can I access the PS+ Cache from the Python API?
Cheers
A
1 -
@AlexanderJHall - Unfortunately, I don't believe the python API supports PS+ Cache yet. It is in the roadmap to do so at some point. I would recommend letting your Solace Account team know that you are interested i this and they can raise the request officially.
@amackenzie FYI
1 -
@AlexanderJHall - As Himanshu correctly pointed out, we do not yet have support for PS+ Cache in our Python API as it is one of the newer APIs. We are constantly working towards API parity between the new and the old so this item is on our roadmap. The current timeline for PS+ Cache support in our Python API is slated towards the end of July and I can certainly update you once this feature is released.
1 -
Thank you @palidhjede 🙏
0 -
Hi @murat,
I forgot to mention you above.
Thanks for the reply. I was thinking of just wrapping the C API to leverage caching, but this could be considerable work. My previous firm did this, but I wanted to avoid this if possible. I'm planning on abstracting the pub / sub logic / imp behind interfaces. This leads to potentially using MQTT w/ retained messages to emulate the caching behavior. I'm concerned about the MQTT overhead though. I experimented w/ MQTT and this appears to work so far, but I don't know all the implications. Just an FYI, my firm is looking at a PubSub+ Cache license but we have mostly python based systems. I will be chatting with our Solace Reps.
Cheers,
Ian
0 -
Hi Ian,
The PS+ Messaging API for Python is itself a wrapper for the C API as you may be aware. It does a lot more than abstract the interfaces though which has lead to some complex problems and slowed down the development on the interface for retrieving data from PS+ Cache. As noted earlier this is now a staffed project but not likely available until the end of the calendar year.
It is certainly a valid approach to write your own wrapper for the C API, but it is as you noted considerable work and would not be compatible with PS+ Messaging API for Python. You would not be able to mix and match on the same broker session, so you couldn't use one for Cache request and the other for messaging. You would need to wrap all the functionality you need.
Please continue to discuss with your reps what is the best solution for you.
Regards,
Ragnar
0
