IoT Storage: Considerations and Recommendations

By CIOReview | Tuesday, May 23, 2017

The first thing that increasingly comes to mind of an IT professional for Internet of Things (IoT) is—a lot of data. For medium and large organizations, on the other hand, IoT is about the potential business outcomes. The reason is simple; more connectivity of devices to the internet will enable the business operation for greater intelligence and bolstered efficiency and effectiveness. At the backbone of this entire phenomenon, there is a less loud, less flashy, and less talked about topic—Storage.

General Electric (GE), also a strong proponent of the industrial IoT, suggests that around 50 billion devices are going to connect to the Internet by 2020. Now, that’s a lot of data generation. Where will it go? What about its storage and protection? Experts around the world want to tame this IoT data deluge but, at the same time, cloud elasticity, data security, and incessant cloud communication need to be addressed. Many of the traditional IT organizations usually plan by estimating their infrastructural (hardware and software) requirements for up to next three to five years. Meeting accuracy of that is difficult; imagine how it will be managed when there is an unpredictable IoT data stream. Also, their devices would need substantial local storage space for both data capture and support of real-time analytics.

IoT Data Storage Considerations

The explosion of data is real. If someone is evaluating to what extent IoT should be embraced, here are 3 important things to consider:

The Significance of IoT data - This untapped pool of data will provide organizations visibility into parts of their operations—driving decisions for businesses. The capability of IoT combined with real-time processing and predictive analytics profoundly enriches management practices by enabling proactive resolution in response to real-time events. Whether it is smartphone data, sensor data, or other endpoint data, the businesses should deploy all of the connected devices to harvest value from the IoT data.

Analyze all of your IoT data – Once done with the storage and protection of data, next is to analyze it. Data becomes more valuable when linked together and with more complex condition sets. Sometimes organization misjudges how much of this data they need to store, which leads to limit their ability to analyze that data. So, that needs to be ignored. One more important factor to look out for is to identify whether the storage could become a bottleneck to any kind of application that supports analytics. For e.g. Hadoop has its own specific scaling and performance requirements for a large number of large files, or a large number of small files of the IoT data.

Branching its possibility out in every direction, it’s still not much clear to know exactly what sorts of practices will be developed in the time to come. To that end, here are three general recommendations:

• Be ready to Scale: When it is about implementations of IoT devices, avoid sizing up the storage requirements at the outset. Instead, watch for a product in the market that can immensely scale to your requirements.

• Define IoT data storage (and backup) tiers: Traditionally, Tier 1, Tier 2, and Tier 3 is for mission-critical, high-performance data, back office applications and user data, and for backup and archive, respectively.

However, with software-defined storage, an organization gains much flexibility. If an organization wants to fully dedicate a cluster just to the IoT data, that’s completely okay. At the same time, if there’s a second thought and you’d want to rather carve out a portion of it and apply specific policies and SLAs to it; that is also possible.

• Distributed  to back IoT storage: The distributed nature of IoT follows the architecture and infrastructure employed to support it. The advantage one can get is to change storage and backup requirements with time as they grow.

What matters at the end is how aligned IoT data storage strategy and implementation is with the organization’s business strategy. Experimenting with IoT justifies only the modest needs. However, in general, it should help an organization expand or collapse storage tiers as needed, all while in support with distributed system.