It’s the time of year for reading the tea leaves. Here are my predictions for the data storage industry landscape in 2021.
The rise of the service mesh
Security remains a vital challenge for new distributed applications that are being deployed across multiple clouds, on-premises data centres and edge locations in existing virtual machines and new container-based services. The legacy zone-based model is too rigid and does not enforce secure protocols. What’s more it seems to be unable to keep up with the growth of remote work.
In response to this, we predict that ‘service mesh’ approaches will be adopted in 2021. These approaches will secure network communication with Transport Layer Security (TLS) as well as authentication and access control using mutual TLS for both workload connectivity as well as toward the edge. This will involve service instance identities with short-lived service accounts and delegation, and more granular authorization using frameworks like SPIFFE. We expect that this will pave the way to ‘zero-trust’ networks, where network policies can be codified and systematically enforced and deployed at fine granularity.
Cloud-native applications and storage infrastructure
Containerization marks a major turning point in the storage industry and will drive a transformation of solution architectures and deployments, with an impact similar to that of server virtualization and the introduction of cloud computing. By 2023 cloud-native approaches will be used to develop and deploy over 500 million digital apps and services, the same number of apps developed in the last 40 years, according to IDC. Enterprise IT is adopting cloud-native and containers as the new principle and model for application development and underlying cloud infrastructure services.
Storage vendors will address this transformation in 2021 with new innovations and solutions to meet the increasing scale and agility demands of container-based services. These new container-centric solutions will converge Container Storage Interface (CSI)-type persistent volumes for traditional data centric applications, as well as object storage and backups, and radically simplify the unique challenges of cloud-native applications.
New generation of flash storage
In 2021, new high-density, high-performance and low-cost flash media will become available. Flash storage has typically been deployed for smaller capacities where low latency was a priority, while use cases with higher volumes of data have turned to high-density spinning disks. Following the introduction of lower-cost, higher-density flash media in 2021, these use cases will naturally gravitate toward capacity-optimised storage that takes advantage of this media to provide benefits in density, scale and agility for multiple workloads.
Object storage and data lakes
As the data lakes market continues to grow at a rapid pace (becoming a $20.1 billion market by 2025, Research and Markets estimates), organizations require a storage layer that makes the data in these gigantic data repositories accessible and useful to analytics applications such as Elastic, Cloudera, Spark, Splunk, and Vertica, and to high-performance file systems such as Weka.
In 2021, we predict object storage will become the predominant storage interface for these analytics applications for a number of reasons:
- Analytics apps can consume data directly using Hadoop-compatible protocols such as S3A. This leverages AWS S3 API, the API of choice for object storage.
- Object stores were designed for semi-structured and unstructured data sets.
- By separating the storage tier and the application compute tier which hosts the analytics application, performance and capacity resources can scale independently.
- Object lock and versioning provide another layer of security.
Hybrid cloud data management will be broadly embraced
The year ahead will see a change in Disaster Recovery (DR) approaches to ensure business continuity strategies for critical data. Hybrid cloud DR solutions that involve a copy of critical data on-premises synchronised with a second copy in the public cloud mean that maintaining and servicing two physical data centres is a thing of the past. This could save organizations thousands, if not millions, of pounds.
Increasing convergence of object and file storage for unstructured data
Unified object and file storage solutions will be broadly adopted in the enterprise in 2021. Cloud-native applications, which naturally consume and interact with object storage over the AWS S3 API, are on the rise. They accompany more traditional applications that rely on file system storage. As organizations simplify their IT environments, they will prioritise solutions that can support large numbers and types of applications.