IoT is on the Rise…
The Internet of Things (IoT) is a hot topic right now, experiencing dramatic growth, driven by the increased use of connected devices, or endpoints. These devices vary from sensors, actuators, and valves in factories, to home automation control systems (Nest, Apple HomeKit, Philips Hue), to wearable fitness trackers (Fitbit, Garmin), and smartphones. What they all have in common is that they generate, monitor, and analyze vast amounts of data, providing instant real-time feedback, and typically, they are found at the edge.
To put this into perspective, Gartner estimates there were 3.96 billion IoT endpoints in 2018 and predicts that number will rise to 5.8 billion endpoints by the end of 2020:
This massive influx of IoT endpoints will lead to an explosion in the amount of data that is created, making storage an integral part of any IoT solution. For datacenters, this is predicted to be in the tens of zettabytes (ZB), and at the edge (outside the datacenter) this is expected to be hundreds of zettabytes worldwide. All of this data will need to be aggregated, processed, stored and managed.
The Need for IoT Edge Computing
IoT devices and sensors constantly generate large amounts of data. As such, traditional centralized or cloud computing models cannot meet processing and analysis requirements in a timely manner. Often a rapid response is required, for example, shutting off or opening a valve in response to a pressure or temperature reading in a chemical manufacturing factory.
These traditional models rely on moving all data from the edge, where the data is created, to a centralized location for processing. This makes them vulnerable to network latency which some applications or processes may not be able to tolerate. In addition, the data generated from thousands of devices can quickly consume all the available network bandwidth capacity.
To address the latency and bandwidth requirements seen within traditional compute models, a new “edge computing” model is required. The main advantage of an IoT edge computing model is being able to analyze the data close to where it is generated and collected. This eliminates network latency and vastly reduces the amount of data being transferred over the wide-area network. It prevents organizations from having to spend considerable sums on high capacity network links and instead focus budget on localized improvements that cost significantly less.
The Advantages of an Edge Computing Model:
Reduce Network Bandwidth
As previously mentioned, IoT devices and sensors generate large amounts of data. For example, offshore oil rigs can generate up to 500GB of sensor data in just a week.
Much of this data is transient in nature, often only useful in a short period after being generated. Therefore, it is not practical to transfer all the data from the edge devices to the cloud. Instead, it is better to analyze or “mine” the data at the edge, only sending the “nuggets” of refined data back to the centralized storage location for further processing (big data). Using this approach significantly reduces the amount of data that needs to traverse the network.
Minimize Network Latency
For time-critical applications that require real-time decision making, milliseconds matter and can be the difference between ensuring normal operations and failure. Transferring data to a cloud or central datacenter for processing and then waiting for the response adds unnecessary network latency. This is even more significant when the network latency from the edge site to the central datastore is high, such as over a satellite link. Therefore it is best to analyze the data close to where it was generated, minimizing the time added by network latency.
Process Data in the Most Appropriate Place
Choosing the location to process or analyze data depends on how fast a decision needs to be made and how transient the data is. For environments that have large amounts of transient data, but are time-sensitive and require immediate feedback, the data processing should be performed at the edge, closer to the devices generating and acting on the data. However, if the data is required for historical analysis for instance and needs to be kept for longer periods of time for regulatory purposes, then it may better to transfer the refined data to a central datastore for offline, post-processing.
Deliver Edge Computing High Availability
For some IoT locations, such as oil rigs, solar farms, or remote processing plants, network connectivity may not have sufficient bandwidth, have high latencies, or be unreliable (high error rates), and in some cases may not even exist. For these locations, using traditional centralized or cloud computing solutions is not an option.
The only option for these locations would be to use an edge computing solution to ensure the smooth running of the service. For very remote locations it would be prudent to ensure that any edge computing solution is highly available. This would confirm that all services and critical data are protected and available in the event of a component failure. This is especially true for sites that may have minimal access to readily available spare parts and IT assistance.
Another factor to consider is the cost of the solution. Using a cloud model may have the benefits of centralizing all the data and relatively cheap compute costs. However, this would require additional network infrastructure to be able to transfer all the data. This, coupled with the huge amounts of data that need to be stored within the cloud, means things could get very expensive, very quickly.
On the other hand, using an edge model would allow data to be stored and processed locally, only sending the important information back to a central location for further analysis. This not only dramatically reduces network costs but also reduces the amount of cloud storage required.
The Ideal IoT Edge Computing Solution
As shown throughout this blog post, a fully centralized solution may not fit within all IoT environments due to network latencies, bandwidth, and/or cost. An alternate approach would be to use a hybrid approach. An edge computing solution is used to process and store the majority of the raw IoT data close to where it was generated, ensuring timely responses to events. Only the important or refined data would be transmitted to the datacenter for further analysis and long-term retention.
To achieve this the edge computing solution needs to fulfill the following requirements:
- Small Footprint: The solution should have a small footprint in terms of space, power, and cooling requirements, as typically there is limited space for IT equipment at each location.
- Robust: IoT solutions are often deployed in harsh environments (i.e. oil rigs, manufacturing plants). Ideal solutions should be designed with robust components. For example, fanless servers for dusty/dirty environments or use of SSDs that are more resilient to excessive vibration, shock, pressure, or extreme temperature changes.
- Highly Available: The solution should be resilient to withstand single component failures, to ensure that operations are not interrupted.
- Low Cost: The solution needs to be low cost for both initial acquisition and ongoing operations. This includes power and cooling, which can be significant as the number of locations requiring edge compute resources increases.
- Simple: The solution should have a simple, repeatable design that makes it easy to deploy and maintain.
- Easy to Manage: The solution should be capable of being monitored and managed from a central support location.
StorMagic SvSAN fits these requirements, making it the perfect IoT edge computing solution. It has been designed with the remote site in mind from the outset and for the realities of poor network reliability often found in remote areas. Starting with just two servers, SvSAN has a lightweight footprint and creates a highly available, virtual SAN using the server’s internal storage, that is both cost-effective and easy to manage. For more information about SvSAN, take a look at our SvSAN Overview and Product Datasheet.