Batch processing in messaging usually means that the code is designed or optimized to process messages in batches instead of one at a time. Usually, the underlying messaging system and the interfaces need to support producing and consuming batches of messages. This makes the system less chatty because data is coming in chunks.
Leave your comments below.
Exciting news, Solace developers and administrators! You can now copy messages between queues!
PubSub+ Event Broker version 10.0.0 has a new management command, copy-message, that lets a broker administrator copy a message from the replay log to a queue, or from one queue to another (like from a DMQ to a client’s queue).
Why is this exciting? Well, when using guaranteed messaging, messages aren’t ever supposed to get lost. But in the real world, things aren’t always that simple. Your consumer might be offline for an extended period of time, causing time-to-live (TTL) to expire for the message. Or there might be an application issue preventing the app from processing the message, resulting in max-redeliveries being exceeded for the message. There could even be a subtle bug in the app, where it acknowledges receipt of the message just a bit too early, before it has fully completed processing of the message (for example, a database write that gets cached in RAM but doesn’t get flushed to disk before a power failure).
And let’s not forget operator error, where an administrator accidentally deletes the wrong message from the queue.
As long as the message is still available in either a dead-message-queue (DMQ) or the Replay log (you are enabling replay for your important message topics, right?), you can now easily get it back. The copy-message command lets you copy the message back to your consumer’s queue!
Copy-message is supported in CLI and SEMPv1 in PubSub+ Event Broker version 10.0.0 and later, and will be supported in PubSub+ Manager and SEMPv2 later this year.
Hopefully, you won’t need this command too often. But if you ever need to restore a message, it’s a great tool to have in your toolkit.
🚀 Solace Academy offers a FREE certification for developers: Solace Certified Developer Practitioner (Level 1)
🎉 As we celebrate the end of Winter and launch into Spring in the northern hemisphere it felt like a perfect time for a challenge and some giveaways! Complete the steps below to be entered in a live giveaway to win!
✅ Here's how you can qualify for the challenge:
1. Take the Certification Exam by March 31st and score 80% or higher on your first attempt for 1 entry. 90% or higher gets 2 entries! Enroll for free here.
2. Reply to this thread with a screenshot of your Certificate and let us know your score.
3. BONUS: You will get an extra raffle entry for sharing your certification badge on LinkedIn with the hashtag #ImSolaceCertified. Don’t forget to tag Solace and include a link to your post in your response.
4. NOBODY LEFT BEHIND!: If you previously passed this certification #3 still applies to you - share on social and you will still get 1 entry into the raffle. **Note that if you won a prize during our December Challenge we want to spread the love to other winners so you can't win again, but we'd still love it if you shared.
💡 Check out this Learning Path to make sure you have all the bases covered before taking the exam
🥇 1st Place: $200 Amazon Gift Card + Free Solace Academy Learning Subscription
🥈 2nd Place: $100 Amazon Gift Card + Free Solace Academy Learning Subscription
🥉 3rd Place: $50 Amazon Gift Card + Free Solace Academy Learning Subscription
🏆 We will do a raffle for the winners on April 5th.
Good luck and happy eventing!
Administrating a Solace PubSub+ account is key to ensuring your services run smoothly. You need to be able to manage users, audit user activity and generate API tokens. This course will teach you everything you need to know to successfully manage your PubSub+ Cloud account.
Who is this course for?
This course is designed for whoever will be administrating your PubSub+ Cloud account. This will be the person responsible for ensuring your users have access and that your account has everything you need to develop and deploy your event driven architecture, so if you're responsible for overseeing your organizations event driven journey, this course is for you.
Here's everything that will be covered in this course:
- Setting up OpenID SSO for PubSub+ Cloud users
- Manually creating users
- Roles and permissions
- API token management
- Understanding service limits
- Understanding audit logs
- and more
What will you know how to do after this course?
After completing this course you will know how to do the following in PubSub+ Cloud:
- Create users and assign them their required roles
- Create Cloud API tokens with the required permissions for external applications
- Set up Single Sign On for federated login
- Read and understand your current service usage and your limits
- Manage your profile and notification settings
- Understand and manage Pre-Authentication
- Understand and monitor Cloud console activities with audit logs
Where can I take this course?
This course is a part of the Solace Learning Subscription at Solace Academy, so if your organization already has a learning subscription this is course is available to you! Learn more about Solace training and learning subscriptions here.
Avro is a common message format used for Kafka records. Large Kafka implementations leveraging Avro often use the Confluent Schema Registry to ensure uniform record schemas across publishers and subscribers.
When using the Solace PubSub+ Kafka Connectors to publish Kafka records as Solace PubSub+ events, it may be necessary to integrate with a schema registry in order to deserialize records. For example, to dynamically synthesize Solace topics using data in a record’s payload. This post covers how to integrate the Solace Kafka Connectors with the Confluent Schema Register using the Confluent Avro Converter, in order to automatically (de)serialize Avro messages using schemas from the Schema Registry.
Procedures to leverage the Kafka Connect Avro Converter for message deserialization:
- Modify the Solace PubSub+ Connector for Kafka: Sink files:
~/pubsubplus-connector-kafka-sink-2.x.x/etc/solace_sink.properties(or solace_sink_properties.json if using json flavor).
- Comment out line 10 of the file using the
- Insert a new line, instructing the connector to use the Avro Converter –
- Insert another new line specifying the location of your Schema Registry Server –
- Save and close the solace_sink.properties file.
- Prepare additional dependencies – To leverage the Avro Converter there are several new java dependencies. We will obtain the majority of them from the Confluent Kafka Connect Avro Converter libraries.
- Unzip the contents of the
confluentinc-kafka-connect-avro-converter-X.X.X.zipfolder to your desired directory on the target machine. For example, we will place them directly off the root:
- Add the
/confluentinc-kafka-connect-avro-converter-7.0.1/libsdirectory to your java
CLASSPATHso the Solace PubSub+ Connector for Kafka: Sink will know where to find the new dependencies. We also have to provide the location of our original Solace Connector lib directory as we blow away, the default process when explicitly setting CLASSPATH.
export CLASSPATH=/kafka/connectors/pubsubplus-connector-kafka-sink-2.1.0/lib/guava-31.0.1-jre.jar:/confluent-hub/confluentinc-kafka-connect-avro-converter-7.0.1/lib/*NOTE: If you have additional jar dependencies add them to the CLASSPATH at this stage.
- Unzip the contents of the
- Add the Guava dependency to the Solace PubSub+ Connector for Kafka: Sink libraries:
- Manually move the
guava-XX.X.X-jre.jarfrom its download location to the
- NOTE: If you would prefer to skip step 2 and manually add the dependencies to the
~/pubsubplus-connector-kafka-sink-2.x.x/lib/directory, these are the additional jars that must be moved:
- Manually move the
- Deploy the customized Solace PubSub+ Connector for Kafka: Sink (
~/pubsubplus-connector-kafka-sink-2.x.x/) to the target system following the Deployment Guide in the Connectors readme.
Once started, the Solace PubSub+ Sink Connector for Kafka reads each record from Kafka using the Avro Converter instead of the generic Byte Array Converter. For each message, the Avro Converter reads the schema ID from the record and obtains the correct schema from the Schema Registry to deserialize the Avro message. The contents of the Avro message can now be parsed and acted upon by a custom RecordProcessor class as described in the Connectors readme.
Role of a "Developer" in building EDA
The role of a Developer in modern times goes beyond the piece of code produced and run in a silo. The resulting module could be the crucial cog in the wheel that keeps business running smoothly. On that basis, considerations on aspects beyond the code and choices around design, architecture, and tools become critical. Above all, the design and implementation to fulfill non-functional requirements start in the development phase itself.
A Developer has to have intimate knowledge of the approach and technique to implement a solution. However, they also need to know macro elements such as architecture, integration modes, employed patterns, and tools - so that delivered code/components augurs well with the rest of the system.
Few things the developers have to absorb before writing the first line of code:
- Practise the "event-driven" thinking
- Familiarize with the planned integration patterns
- Master the message-exchange patterns
- Make the best use of features, tools, and frameworks of the EDA platform
This post consolidates useful resources available on various Solace sites (website, blog, documentation, and youtube) for you to review and familiarize anytime during the preparation for Solace Developer Practitioner certification.
Solace Academy offers numerous byte-sized training modules on specific topics that can help you as a Developer to explore Solace PubSub+ Platform, components and features. Here is a list of such modules for your reference.
- Introduction to Solace Technology (ID: E-J0E7QV)
- Solace Essentials (ID: E-7VRG21)
- Being Event-Driven (ID: E-602RL1)
- Event Driven Architecture Foundations (ID: E-P086EV)
- PubSub+ Event Broker Key Features (ID: E-XVDL90)
- Fundamentals of Solace Development (ID: E-XVDGPV)
- Shared Subscription by Example (MQTT & SMF) (ID: E-D19440)
- Dynamic Message Routing (DMR) (ID: E-N05QZV)
- Replay Intro - Whiteboard Session (ID: E-Y1QWZ1)
- Guaranteed Messaging Overview (ID: E-7VR271)
- Queue Durability (ID: E-6VXM71)
- Queue Consumer Pattern (ID: E-YVMJQV)
- How to Build an Event Mesh with Solace PubSub+ (ID: E-P08J60)
- PubSub+ Monitor - A Technical Overview (ID: E-6VXY71)
Developer Journey to EDA
The following document captures the first-hand experience of a developer's journey into EDA - covering origins, evolution, and the current state. This document is a good starting point and a must-read.
Mastering Solace Essentials
👀 Whats the news?
We've been working on a real treat for our developer community here at Solace over the past couple of months and that is --> drum roll please -->
The new Solace PubSub+ Messaging API for Go!
We will be releasing an early access to members of our community who are interested to use the bling new Go API .
⚙️ What features will be included in the early access?
Below are the features that are included in the Solace PubSub+ Messaging API for Go v0.9.0
- Direct Messaging (Publish, Receive, Subscribe/Unsubscribe)
- Persistent Messaging (Publish, Bind to Queue, Durable Endpoint Creation, Add/Remove Topic Subscription, Trigger Message Replay,Selectors)
- Broker Connectivity support ( SSL/TLS, Compression )
- Authentication: (Basic, Client Cert, OAUTH, Kerberos)
🤝 How can I take part?
Good question! Simply comment on the this community post here to stay up to date with the latest news on this new release. You can also let us know what projects you would like to use it in, any wish list, thoughts, or just general comments and questions you have about the API.
🏠 Cut the chase: Where can I download the API from?
Join this group! We will be uploading the archive API there for users to use and download 👇 https://solace.community/group/4-solace-early-access-golang-api
🛠 Samples; where are they?
We will be updating this post with GoLang samples! Stay tuned 🙌
🔊 What's in it for me? Why you should join this group
Being part of this EA group, you will have access to the following perks
- Swag when you give feedback and contribution
- Opportunity to give feedback on the API and shape the direction
- Collective knowledge from all the members of the group
- Tips and Tricks on how to use the API more efficiently
- Improve your GoLang programming skills!
Stay Tuned !