Introduction
Throughout 2019, our research surveys and discussions with business and IT leaders revealed that cloud-native technologies have become their top priorities as they strive to become digital businesses. Cloud-native architecture is a departure from monolithic applications and waterfall release processes. The technologies that enable it are desired for their speed, efficiency, and how they support the distribution and portability of applications and services across hybrid infrastructure that include public/private clouds and on-premises environments.
The 451 Take
Cloud-native approaches to software and service design enable enterprises to act faster, more efficiently and at greater scale. Re-platforming to cloud-native is therefore an imperative for digital transformation strategies, and it will sweep through the market over the next decade, much like the re-platforming to the internet and web in the 1990s and 2000s. We expect the cloud-native trend to continue to grow, fueled in part by intersections with adjacent technologies and trends, including data and analytics, AI and ML, security and IoT/edge computing – all of which play a role in facilitating digital transformation. We also expect the cloud-native market – populated by a burgeoning number of startups, as well as established giants – to undergo consolidation, and trigger several waves of M&A activity as vendors seek to gain talent and market share.
Cloud-Native Defined
Cloud-native technologies are used in the design or redesign of applications built to run in public, private and hybrid cloud infrastructure. They include containers, service meshes, microservices and serverless functions, all of which can be independently updated, controlled, scaled or reconfigured to deliver a coordinated application experience.
Why Cloud-Native?
Cloud-native is the latest in an evolutionary trend toward greater efficiency and flexibility in applications development and deployment. As we examine the history of application development (see Figure 1 below), each successive technique enables greater degrees of automation to gradually reduce the need to manage and deploy the IT infrastructure necessary to run applications.
Waterfall techniques represented the 'ye olde architecture of yore' – where applications where upgraded in up-to-18-month cycles, and ran on servers in datacenters. Agile techniques accelerated development cycles to weeks, and sometimes days. Operations often lagged though, and delayed deployment. Virtualization and IaaS offerings from cloud service providers helped. Their use drove development and operations silos within organizations closer together.
Then DevOps was born that enabled a more integrated IT organization capable of more rapid deployment using continuous integration/continuous deployment (CI/CD) technology. Speed and workload portability across distributed infrastructure became a priority driving application containerization. Cloud-native application development ensued.
Each successive stage in this continuum brought greater degrees of IT process automation of all types. The need to understand the configuration, capabilities and capacities of compute, storage and network resources gradually diminished. In cloud-native architecture, infrastructure became nearly invisible to the developer. Their priorities could then be focused on codifying business logic and processes.
Without concern for underlying infrastructure, changes can be made rapidly, code can move freely (almost) across a variety of distributed execution venues on-premises and in a variety of clouds. Enterprises have greater ability to adapt to changing markets, the needs of customers, and the actions of rivals and new market entrants.
Adoption
Many organizations have embraced, and many others are considering, cloud-native technologies. To better understand its progress, we surveyed over 400 business and IT professionals about their rates of adoption of the core enabling technologies of cloud-native computing. Figure 2 below illustrates that adoption is already well entrenched in many organizations, and that the pace will be accelerating in the next 12-24 months.
Market Research Coverage
The state of cloud-native computing, however, is not necessarily settled science at this time. Much innovation is currently under way in a variety of technology market sectors. The Cloud Native Computing Foundation (CNCF) has crafted a technology and vendor landscape to help bring order and awareness to all its moving parts. However, it changes rapidly, and currently describes 31 discrete technology market sectors.
Our research of cloud-native technologies is patterned after the CNCF. But our focus for 2020 will concentrate on nine specific technology market sectors, and include examination of four infrastructure and services sectors. We believe these to be essential current priorities for those considering cloud-native computing.
AppDev, DevOps, CI/CD
Our research and conversations highlight how cloud-native is an integral part of enterprise DevOps and CI/CD releases that are focused on faster deployment and more efficient operations. Our Voice of the Enterprise: DevOps H2 2019 survey indicated that 90% of enterprise DevOps teams deem open source 'very or somewhat important' to their DevOps deployments.
In that survey, respondents indicated that more than half of developers, IT operators and combined DevOps teams use software such as containers and Kubernetes to speed developer onboarding and deployment, drive productivity, and streamline and scale operations while maintaining performance and security. While the bulk of enterprise CI/CD pipelines today are built in on-premises and private cloud environments, there continues to be interest and growing use of SaaS and IaaS, particularly as more enterprises put more DevOps on to public clouds.
Using cloud-native technologies and methodologies such as containers, microservices and Kubernetes, enterprises are better able to abstract and automate the toil of IT administration so developers can focus on new features, products and innovation, while IT operations units can manage larger-scale infrastructures with smaller teams. Representative vendors in this sector include Anchore, Atlassian, Atomist, AWS, Chef, CircleCI, CloudBees, CodeFresh, Google, Hashicorp, IBM/Red Hat, JFrog, Microsoft, Plutora, Pulumi, Puppet, VMware/Pivotal, Weaveworks and XebiaLabs.
Data Management, AI, Analytics
The rising importance of cloud-native is an opportunity and challenge for existing data, AI and analytics vendors, and many are still working to get their data processing technologies to play nicely with cloud-native infrastructure elements such as containers and serverless. The orchestration capabilities of platforms like Kubernetes are becoming increasingly popular with next-generation workloads. For example, in the AI space, Kubernetes can be used to scale up training and inference deployments across multiple cloud environments.
The successful use of cloud-native technologies will give innovators important benefits, such as faster provisioning, scaling and the potential for improved cost efficiency and workload portability across multiple cloud environments. Nevertheless, there are still many gaps that need to be addressed to ensure the transition to cloud-native goes smoothly. Data management vendors in particular have done relatively little to make themselves appealing in this space, although with persistent storage for container-based applications gradually becoming the norm, these vendors will need to adapt to manage the data in these new platforms.
Turning to 2020, we will be following the evolution of cloud-native databases, analytics and AI platforms, given these platforms leverage containers, microservices, service meshes and other technologies. Representative vendors we will be covering in 2020 will include AWS, Couchbase, Google, IBM, Ignazio, NuoDB, NVIDIA, Snowflake and Qubole.
Orchestration and Management
Cloud-native management and orchestration now means Kubernetes. Our Voice of the Enterprise: DevOps H2 2019 survey of 500 IT decision-makers and practitioners, primarily in North America, indicates that more than 70% of enterprises are standardizing on Kubernetes over the next one to three years.
Another 19% say it will take them more than three years to get there. Few technologies, if any beyond the hyperscale public clouds, are as influential on modern IT operational approaches as Kubernetes. In addition to its use for container management and orchestration, Kubernetes is also a distributed applications platform.
This is well-timed, with enterprise hybrid and multicloud deployments that require portability across different environments, and we expect this will keep Kubernetes adoption growing over time. Kubernetes is also illustrative of the modularity and extensibility of cloud-native software with the concept of 'operators,' which are software packages to support and simplify a range of integrations and enhancements to Kubernetes for databases, load balancing, monitoring, security and many other capabilities.
Representative vendors that we will examine in this sector include AgileStacks, Alibaba, AWS, D2iQ (formerly Mesosphere), Digital Ocean, Google, IBM/Red Hat, Microsoft, Mirantis, Huawei, Platform 9, Rancher, SUSE and VMware/Pivotal.
Service Mesh
There's a lot of excitement around service mesh, and with good reason. As microservices push software development and execution to become more granular and distributed, new ways of authenticating and controlling service-to-service communications are needed. Rigid and lengthy release pipelines for traditional software are naturally giving way to more nimble, lightweight and flexible routines.
But the transition can create complexity and risk. To date, service mesh implementation has been difficult due to fiddly configuration and management demands. Add to this the competing control plane options (e.g., Istio, Consul, Kuma, Linkerd, NSX and AWS's proprietary App Mesh) at various stages of adoption and maturity, and you get a perfect storm of confusion – dare we say a bit of a 'service mess.' Service mesh shows promise for bringing observability, traffic management and policy control to modern-day runtime workflows, but this is an emerging opportunity with major decisions still to be made.
In 2020, we will be tracking enterprise adoption of service mesh, lessons learned by practitioners and providers – including those focused on extending an application-level service fabric across hybrid environments – and progress in creating a federated 'mesh of meshes' to negotiate between the various alternatives. Vendors covered will include AWS, Microsoft, Google, IBM/Red Hat, VMware, Buoyant, Hashicorp, Solo.io, Aspen Mesh, Netifi, Containous and Tetrate.
Networking, IoT, Edge
The variety of network connectivity environments continues to expand as organizations distribute applications far and wide. Within the datacenter, high-speed SDNs interconnect physical and virtual servers and container pods, while interconnectivity between clouds and to the edge creates a diverse operating environment and set of demands.
For example, container environments have their own networking stacks independent of the rest of the application environment, while hypervisor systems may have a different network stack, and cloud services yet another. They are tied together over the WAN. Depending on the IoT or edge devices, there may be even more network technologies coming into play. Network automation can tame much of the ensuing complexity of interconnecting applications, servers, clients and things, but will take foundational, up-front work by enterprises for success.
Established networking infrastructure vendors include Arista, Aruba, Cisco, Dell EMC, Huawei, Juniper and VMware, as well as startups that could disrupt the status quo like Arrcus, Pensando and Volterra. In addition to the infrastructure vendors that are offering network telemetry features, vendors like Broadcom (CA Technologies), cPacket, Gigamon, Ixia, Kaloom, NetScout, and Nubeva collect, process and report on network traffic in a variety of environments. Tying it all together are multivendor network automation systems from companies like Anuta Networks, Apstra, Gluware and Itential that aim to rein in the chaos and automate workflows across vendors and environments.
Storage
The use of cloud-native technologies presents new challenges for both storage vendors and enterprise end users. The lightweight nature of containers can introduce cost savings by necessitating less capacity; however, maintaining the persistence of the data volumes for stateful applications built on containers can be complicated, especially at scale, if an organization does not want to compromise some of the inherent benefits of containers, such as workload portability.
Through orchestration technologies like Kubernetes, organizations can achieve dynamic storage provisioning and scaling; and with accompanying software like CSI drivers, statefulness can be ensured. On the vendor side, storage suppliers are now tasked with supporting these cloud-native technologies, including containerizing their own storage platforms and providing integrations for orchestrators.
Additionally, the next frontier, after ensuring data persistence, is providing the ability to protect and manage those applications and their data. In the coming year, we can expect to see an even greater emphasis put on backup, disaster recovery and migration capabilities for cloud-native apps.
In 2020, we will be looking at veteran and startup storage vendors as the former adapt their portfolios to the needs of cloud-native apps, and the latter seek to establish footholds in the nascent sector. This will include vendors such as Netapp, Hewlett Packard Enterprise, Dell EMC, IBM, Hitachi Vantara, Pure Storage, Red Hat, Diamanti, StorageOS, Kasten, MayaData, ROBIN and Portworx.
Serverless
The promise of serverless applications – where reusable functions and triggers are assembled into software that works independently of the infrastructure that executes it – has attracted a wave of startups, open source projects and cloud providers. The potential benefits are too compelling to ignore: faster development, hands-off provisioning and dramatically lower costs. Applications invoke compute resources only when needed, operating on a pay-per-use rather than pay-per-provision basis as virtualized hardware does.
Although the technology for building and operating serverless applications at scale still has rough edges, partly due to the lack of an open standard (CNCF is working on this), approaches such as Knative (serverless for Kubernetes) are accelerating adoption. Companies are using 'serverless first' strategies for applications, given the favorable economics of this new managed service paradigm. The key question is how much compute will ultimately go serverless.
Through 2020 we will see the rise of cross-cloud serverless, integration (bringing serverless to legacy apps), serverless programing languages and the provision of state to serverless approaches. Representative serverless vendors covered in the 2020 will include AWS, Microsoft, Google, IBM/Red Hat, VMware, Serverless, Neweba, Oracle, Huawei, Triggermesh, Pipegears, NowFloats, Xqiz.it, Spotinst and Alibaba.
Observability
Cloud-native application environments are impacting application performance and network performance management strategies as vendors adapt their existing virtual and agent-based architectures to the new environments. Applications deployed in dynamic environments like containers, microservices, serverless and the cloud require new approaches to monitoring. We're observing some overlap, in terms of value delivered to users by vendors from historically distinct monitoring categories.
In particular, we're seeing some vendors employ techniques familiar to network performance monitoring (NPM), infrastructure monitoring and application performance (APM) monitoring that, while collecting and analyzing different data sets, deliver insight that solves some of the same sets of problems for users. We anticipate continued efforts by both NPM and infrastructure monitoring vendors to deliver application insight in ways that might steal some market share from traditional APM vendors by highlighting application interconnection and location-independent data.
However, some of the specific capabilities for cloud and microservices environments are truly differentiating, and the specific product features will be more important for new customers adding net-new network and application data collection to their environments. The new capabilities should strengthen existing customer commitment to incumbent vendors as enterprises migrate applications – and monitoring – to cloud-native architectures. Representative vendors covered in 2020 include BMC Software, Broadcom (CA Technologies, Cisco ), AppDynamics, Datadog, New Relic, SolarWinds, Splunk, Big Panda and Dynatrace.
Security
With many organizations consistently indicating that security is one of their key concerns on anything cloud related, it is no surprise that the area has tremendous interest from not only security vendors looking to address customer needs, but also from the cloud platforms themselves, all adding significant security guidance and functionality.
This creates opportunities and pressures. For customers that are able to grasp the capabilities being offered by providers and insert them into their security practices, there's a potential reward of better security outcomes at a reduced cost (both in terms of dollars as well as friction). For vendors, it creates pressure to deliver and demonstrate the value above and beyond what the platform offers. Specifically, for cloud-native security, the increased security capabilities of Kubernetes deployments, service meshes, and the reduced footprint of serverless function execution are good examples of these trends.
Throughout 2020, we expect to follow the evolution of this sector, covering a variety of stakeholders, including key capabilities from AWS, Microsoft Azure and Google Cloud, but also vendors from the security industry, including well-known providers such as Palo Alto Networks, Check Point, Trend Micro and VMware, as well as newer entrants like Aqua Security, StackRox, Tigera, Octarine, NeuVector and Styra.
Infrastructure and Services
This represents a 'catch all' market sector to examine the dynamic landscape of hybrid IT infrastructure execution environments, suppliers and specialists, which provide the engines and highways for container traffic. These entities need not have specialized cloud-native offerings to participate in cloud-native ecosystems, but most will, indirectly. We organize the study of this sector into four subsectors: cloud services providers; hosted, managed services providers; private clouds; and training and certification.
Cloud services providers build and rent the physical storage, network and compute infrastructure on which cloud-native application containers are run. Managed services providers add specialized value on top of their own or partners' infrastructure to meet specific client or sector needs. Some managed services providers may offer to build or run private cloud infrastructure on behalf of clients, or private clouds may be owned and operated directly by an enterprise.
Private cloud is presented as a discrete sector, acknowledging that while the long-term destination for cloud-native applications will be public clouds, many organizations will maintain control of private clouds or make use of managed service providers as part of their IT environment for any number of reasons.
Finally, training and certification suppliers exist to ensure that cloud-native ecosystems are well supported in terms of the necessary technical and process knowledge transfer, and to help increase enterprise overall cloud-native readiness. Training and certification is critical to reduce the 'time to enlighten' those responsible for executing cloud-native strategies, and in building scale around a considerable effort to raise entire IT ecosystems' IQ. We currently cover nearly 200 vendors across these subsectors.
Conclusion
The IT industry is moving toward containers, microservices, Kubernetes, serverless and other cloud-native technologies. In fact, most organizations are already working with them at some level, exploring what new outcomes can be achieved. They do so because cloud-native enables organizations to access speed and agility that was not previously available. It will, though, require new knowledge and organizational approaches to IT strategy and applications development.
As organizations seek new digital services and experiences, they will need to continuously augment their software IQ to successfully compete in an aggressive digital economy. We believe that consistent study of cloud-native technologies and cloud operating, and delivery, models must be the basis for this intelligence.
In the near term, driven largely by digital transformation and the need to embrace and leverage new technology, cloud-native approaches will more deeply permeate large enterprise organizations. Similar to the DevOps trend, this means increasingly pulling in additional stakeholders, including administrators and line-of-business leaders. Cloud-native technology and methodology will probably follow the pattern of agile and DevOps to reach half or more of organizations within the next few years.
It is also important to note that the concept of cloud-native was meant to mean more than containers, Kubernetes or serverless, leaving room for the next technology. This may be a combination of existing ones – integration with adjacent trends, such as DevSecOps, data analytics, AI and ML – or something currently unknown.
NOTE: the full list of authors for this report include Jay Lyman, Henry Baltazar, Fernando Montenegro, Jean Atelsek, Mike Fratto, Liam Rogers and Nancy Gohring.
Carl Lehmann is a Principal Analyst in the Applied Infrastructure & DevOps research channel. He leads 451 Research's coverage of process automation and integration in hybrid IT architectures, as well as how hybrid IT affects business strategy and operations. The markets covered in his research include digital automation platforms (including workflow and business process management suites), robotic process automation technology, process discovery and mining technology, and hybrid integration platforms (including integration PaaS and API management).
