Remember when downloading a movie took hours to complete? At the time, it seemed like a reasonable thing. Movies are big files, and dial-up or DSL speeds were only so fast. But today? A Netflix movie that needs to buffer at all is cause for anguish. The same thing is true about manufacturing systems, enterprise applications, network infrastructure, and Smart Cities projects. In the past, it was sufficient to collect massive swaths of data for post hoc analysis and then generate time series reports, because that was the expectation. However, as real-world infrastructure and other data sources are brought online, it’s clear that there is untapped value to be captured from analysis and correlation of real-time data streams.
I love spending time with analysts / data scientists / quants. It takes me back to my days as a grad student when gnuplot was just becoming a thing. It’s still cool, and I’m still amazed at the rich tools that are freely available that enable data owners to quickly get a handle on its meaning.
Companies in a variety of industries are attempting to leverage data being generated at the edge. Whether it’s related to equipment, motion, position, the environment or other types of information, the data is often vast and is generated at speed. Traditional approaches have been to manage connectivity at the edge, while performing any complex analysis or learning in the cloud. Along the way to being useful in a cloud application, the data must be collected, transmitted, transformed, processed, filtered, stored, and analyzed. This creates a variety of challenges, especially if the goal is not to react from the data, but to make decisions in real time.
Enterprises and public sector organizations are drowning in real-time data from equipment, assets, suppliers, employees, customers, and city and utility data sources. The hidden insights buried within this data has the potential to optimize production, transform efficiency, and streamline flows of goods and services - but finding insights cost effectively remains a challenge. It is important to cost-effectively learn on data at the “edge” as it is produced. Enterprises need an architecture for learning on time-series data at the edge, using commodity hardware to create an efficient fabric of edge devices.
Businesses are rapidly adding new devices, tags, sensors, automation, software and compute elements to help track and optimize business performance – but are rarely able to act on the data generated by them. Centralized cloud systems struggle to scale when enterprise applications aggregate data from multiple local networks, and then must sync state throughout the application. Enterprises want to improve efficiency, cut costs, prevent failures, track assets and improve safety – driving a need to analyze and act on streams of data from their assets, customers, users, infrastructure and operations. However, the cloud-first paradigm is insufficient to deliver maximum value from data analytics investments. The time and cost of shifting data to a central location, storing it, and writing new apps to analyze it before a decision can be taken results in only a small subset of new data being made available, and often delivers insights too late to act on them.
The amount of edge data being generated globally is growing exponentially, creating both opportunity and challenges for data-rich enterprises. While analytics technologies continue to improve, IDC Research predicts that only 15% of data will be usefully tagged by the year 2025. In order to maximize the value of enterprise data, software must move to the data source. Making intelligence capabilities available at the edge can realize massive efficiency gains, and significantly lower operating costs for edge applications.
Big Data was never going to be a universal panacea. The goals of Big Data are clear, monitor and measure enterprise business and processes, analyse and act on that data to achieve ever higher levels of efficiency and cost reductions. But it’s prohibitively costly to store every bit of data generated by a business today, especially considering that the vast majority of enterprise data being created carries little or no value. There are many insights to gain within the mountains of data being created daily, but enterprises must employ new strategies which identify actionable insights in real-time economically from streams of dark data.
You’ve come to the conclusion that your business needs an edge computing strategy. You’re buried under a mountain of sensor data and your existing business systems just can’t keep up. Edge computing will enable you to take advantage of real-time analytics to reduce costs, prevent equipment failure, and improve visibility into business processes, but you’re just not sure how to get started.
We've put together this Infographic to share our learnings on edge computing for massive real-time apps. Edge Computing enables massive scale Smart Cities applications. Distributing application intelligence to each Edge Device allows Smart City applications to dynamically compose multiple data sources into real-time experiences.
One of the biggest challenges with monitoring the health of industrial equipment is in transforming torrents of raw, unstructured machine data into useful insights about maintenance needs, system performance, and anomaly detection. Traditional methods of “store everything now, ask later” can overwhelm networks and expose bottlenecks which delay the discovery of insights until it is too late. In order to capture insights about machine health and react to those insights in a timely manner, it is critical that machine data be processed at the edge.