History shapes the Future – but EVENTS stitches the two
Source: Gartner “Stream Processing: The New Data Processing Paradigm” 9 April 2019, Sumit Pal
A real-time data (aka event) would eventually become stale, historical data. Is there more to gain from Events than the historical data? I think so!
My thought on Events and the influence they wield in the modern systems' design – centered and inspired by the above Gartner diagram. The time-value-data curve demonstrates the value of data on the axis of availability. It is clear that the more the delay the lesser the value of data in the decision-making.
What is PAST?
When the data availability spans Months, Days, and Hours – rightfully they are referred to as ‘Historical Data’. You are left with just data, simply the last state(s) of systems, records, objects, or resources - and for good reasons none of the runtime and intermediate changes would have been captured. This leads to numerous disciplines in the “data processing” domain such as Data Warehouse, Data Lake, Data Analytics, Big Data Analytics, Machine Learning, Time-Series Analysis, and so on.
What about PRESENT?
When the data availability is real-time, the name of the games becomes ‘Events’ – an encapsulation of data and the associated context in which they came into existence. This naturally opens up more powerful and much desired techniques like Predictive and Preventive Analytics, Actionable Insights, Change Data Capture, Propagation, and more.
What of FUTURE?
Though the learning from the historical data is useful, it still represents the PAST. It can guide but not guardrail the journey into the future. The data from the PRESENT as they occur in real-time, referred to as events - can shape the PRESENT as well as influence the FUTURE. This is true especially in the operational aspects around availability, scaling, utilization, and insights, with the potential for a real-time feedback loop for course corrections.
Events and Event-driven systems are well suited for building future-ready solutions and the collected real-time data shall gracefully age and still be utilized by the historical data processing systems.
Present shall lead to the Future, while it gracefully ages to become the Past.
Comments
-
This is great stuff.
Shows how important quick analysis is before data becomes old and irrelevant to certain extent.2
