IoT is on the rise . . .
The Internet of Things (IoT) is a hot topic right now, experiencing dramatic growth driven by the increasing number of devices being used, from sensors, actuators and valves in a chemical factory to home automation control systems (Nest, Apple HomeKit, Phillips Hue) to wearable fitness trackers (Fitbit, Garmin) and smart phones. What they all have in common is that they all generate, monitor and analyze vast amounts of data, providing instant real-time feedback.
To put this into perspective Gartner estimates that are 8.4 Billion connected devices in 2017 and predicts that this will rise to 20.4 Billion devices by 2020:
In addition to the massive growth in connected devices, this will also lead to an explosion in the amount of data that is created making storage an integral part of any IoT solution. For data centers, this is predicted to be in the 10’s of Zettabytes (ZB), at the edge (outside the data center) this is expected to be 100’s of Zettabytes worldwide, all of which need to be aggregated, processed, stored and managed.
The need for IoT edge computing
IoT devices/sensors constantly generate large amounts of data, as such, traditional centralized or cloud computing models cannot meet the processing and analysis requirements in a timely manner, as often a rapid response is required, for example shutting off or opening a valve, in response to a pressure or temperature reading in a chemical manufacturing factory.
The reason for this is that these models rely on moving all data from the edge, where the data is created to a centralized location for processing, adding network latency, in addition to this, the data generated from thousands of devices which can quickly consume all the available network bandwidth capacity.
To address the latency and bandwidth requirements seen with the traditional compute models, a new “edge computing” model is required. The main advantage of an IoT edge computing model is to analyze the data close to where it is generated and collected, eliminating network latency and vastly reducing the amount of data being transferred over the wide-area network.
The advantages of an edge computing model:
Reduce network bandwidth
IoT devices and sensors as previously mentioned generate large amounts of data (Offshore oil rigs can generate up to 500GB of sensor data weekly), much of which is transient in nature, e.g only useful in a short period after being generated.
Therefore it is not practical to transfer all the data from the edge devices to the cloud. Instead, it is better to analyze or “mine” the data at the edge, only sending the “nuggets” of refined data back to the centralized storage location for further processing (big data). Using this approach significantly reduces the amount of data that needs to traverse the network.
Minimize network latency
For time-critical applications that require real-time decision making, then milliseconds matter and can be the difference between ensuring normal operations and failure. Transferring the data to a cloud or central data center for processing, then waiting for the response, adds unnecessary network latency, this is even more significant when the network latency from the edge site to the central data store is high, such as over a satellite link. Therefore it is best to analyze the data close to where it was generated, minimizing the time added by the network latency.
Process data in the most appropriate place
The location where to process or analyze the data depends on how fast a decision is needed and how transient the data is. For environments that have large amounts of transient, but are time-sensitive, requiring immediate feedback, then the data processing should be performed at the edge, closer to the devices generating and acting on the data. However, if the data is required for historical analysis for instance and needs to be kept for longer periods of time for regulatory purposes, then it may better to transfer the refined data to a central datastore for offline, post-processing.
Deliver Edge Computing Availability
For some IoT locations such as oil rigs, solar farms or remote processing plants, the network connectivity may not have sufficient bandwidth, have high latencies, or may be unreliable (high error rates) and in some cases may not even exist. For these locations using traditional centralized or cloud computing solution is not an option.
For these locations, the only solution would be to use an edge computing solution to ensure the smooth running of the service. For very remote locations it would be prudent to ensure that any edge computing solution is highly available to ensure that all services and critical data are protected and available in the event of a component failure – especially for sites that may have minimal access to readily available spare parts and IT assistance.
Another factor to consider is the cost of the solution. Using a cloud compute model may have the benefits of centralizing all the data and relative cheap compute costs. However, this would require additional network infrastructure to be able to transfer all the data, this coupled with the huge amounts of data that need to be stored the cloud storage costs could get very expensive, very quickly.
On the other hand, using an edge compute model, would allow the data to be stored and processed locally, only sending the important information back to a central location for further analysis, this not only dramatically reduces the network costs but also reduces the amount of cloud storage required.
The ideal edge computing solution
As shown throughout this blog post, a fully centralized solution may not fit with all IoT environments due to network latencies, bandwidth or cost. An alternate approach would be to use a hybrid approach and edge computing solution is used to process and store the majority of the raw IoT data close to where it was generated, ensuring timely responses to events, with only the important or refined data being transmitted to the data center for further analysis and long-term retention.
To achieve this the edge computing solution needs to fulfill the following requirements:
- Small footprint: The solution should have a small footprint in terms of space, power and cooling requirements as typically there is limited space for IT equipment at each location.
- Robust: It is highly likely that IoT solutions are deployed in harsh environments (oil rigs, manufacturing plants). Any solution should be be designed to use robust components, for example fanless servers for dusty/dirty environments or use SSDs that are more resilient to excessive vibration, shock, pressure or extreme temperature changes.
- Highly Available: The solution should be resilient to withstand single component failures to ensure that operations are not interrupted.
- Low cost: The solution needs to low cost, this is for both initial acquisition costs and the ongoing operations costs that includes power and cooling, this can be signigicant as the number of locations requiring edge compute resources increases.
- Simple: The solution should have a simple, repeatable design that makes it easy to deploy and maintain.
- Easy to manage: The solution should be capable of being monitored and managed from a central support location.
SvSAN fits these requirements, making it the perfect IoT edge computing solution. It has been designed with the remote site in mind from the outset and for the realities of poor network reliability often found in remote areas. Starting with just two servers, SvSAN has a lightweight footprint and creates a highly available, virtual SAN using the server’s internal storage, that is both cost-effective and easy to manage. Click here or check out the links in the sidebar for more information about SvSAN!