The Evolution of Infrastructure Markets — Cloud Computing
unpacking how new markets evolved around AWS
Today, the venture world has its eyes turned towards AI, specifically large language models (LLMs) and generative AI. There are a handful of large infrastructure businesses like OpenAI, Cohere, Stability AI, but it still feels like early days for how this broader ecosystem evolves. A couple of questions continuously comes up — does value accrue at infra or app layer? Is this all going to be a commodity? what about open source?
In trying to answer these questions for myself, I spent some time digging into the maturation of the cloud computing market. I imagine similar questions were raised around commoditization and where new startups could be created. The following is a simplistic version of how we went from storage and compute primitives to the far more complex cloud ecosystem we have today.
Following this post, I’ll share what I think this evolution could look like applied to the LLM and generative AI inflection point we’re seeing now.
The Beginning
Large infrastructure markets are created when two things happen:
An enabling technology 10x’s its predecessors
A demand catalyst creates a similar problem for a wide range of end applications
When both preconditions are fulfilled, large infrastructure companies can be built to provide an abstraction between the technology and the use cases it unlocks.
To put this in more concrete terms, let’s look at the birth of cloud computing, one of the largest markets in technology today.
The enabling technology was the Xen virtualization engine, which allowed for the pooling and utilization of physical server resources defined by software alone. This allowed software to manipulate and partition the physical server world more effectively.
The demand catalyst was the widespread adoption of the internet. This gave way to internet-scale businesses like Facebook, Twitter, Pinterest, Dropbox. All these businesses would face the similar problem of scaling their compute and would turn to Amazon Web Services (AWS)1 to provide this infrastructure.
These two events created the possibility of a cloud computing market. AWS capitalized by providing the right level of abstraction with their compute and storage primitives. As a result, companies no longer needed to build and manage complex physical data centers. This is somewhat of an oversimplification as cloud computing as we know it is just one step in a broader trend around virtualization.
Keep in mind, this market seems obvious today, but it wasn’t in 2006.
Abstractions
After AWS provided their first primitives and catalyzed what we now know as the Infrastructure-as-a-Service market, further abstractions were developed. Just a year later, Heroku launched its Platform-as-a-Service (PaaS) product, providing a further abstraction on top of AWS cloud primitives to make deployment easier. Today, there are serverless products like AWS Lambda that aim to remove deployment as a consideration altogether. Without getting into much detail, each further abstraction aimed to remove the customer burden of dealing with the cloud themselves.
Notably, each successive abstraction was not objectively better than the last. Each iteration after IaaS came with performance and cost tradeoffs that ultimately limited its addressable market.
Without a doubt, the dominant abstraction is IaaS today. It’s where the majority of dollars are spent and are the moneymakers for Amazon, Google, and Microsoft. It has become more challenging for startups to win purely at the abstraction level, but changes in the edge computing and serverless ecosystems are opening a window today for new players.
insight 1: There is a “correct” level of abstraction. It’s a combination of ease-of-use, performance, and cost that each infrastructure business needs to balance.
Workflows
Following these abstractions, we see an explosion of services that are ancillary to the compute and storage primitives that AWS first provided. This next wave of companies solved 1) cloud-related “workflow” problems or 2) new cloud-adjacent infra problems.
Cloud Workflow Problems
AWS started as an “abstraction” company. The new technology primitives they brought to market required a new set of workflows around provisioning and managing cloud infrastructure. As companies adopted the cloud, we saw new categories and tools emerge.
DevOps
Companies like Hashicorp entered the market with infrastructure automation products. Rather than providing an abstraction of a specific resource, their Terraform product simplifies the provisioning and management workflows of infrastructure. Hashicorp is the exact embodiment of the workflow-centric approach and it’s why “Workflows, not technologies” is one of their company pillars.
Companies like GitHub2 and GitLab emerged to help developers more easily manage their Continuous Integration and Continuous Deployment (CI/CD) processes. The elasticity of the cloud made continuous deployment more attainable, but still challenging without the right set of tools. These startups won over developers early in the development process and extended further into cloud deployment.
Monitoring
Companies like Datadog and New Relic provide performance monitoring and logging solutions which is primarily a post-deployment problem. As applications became highly scalable, monitoring became challenging with legacy tools. By building a cloud-neutral approach, these companies were able to focus on the monitoring-centric workflow.
insight 2: The cloud workflow vendors benefitted from remaining cloud-neutral. They were able to embed themselves into complex workflows and were preferred to cloud-specific vendors early on.
Cloud-Native Problems
Then there’s a number of companies that are one step further removed from the cloud. As a result, they typically require a different level of expertise and often, a user persona that isn’t the focus of AWS. These problems were far enough outside of AWS’ initial focus to allow for new startups to be built.
AI/ML
AI/ML benefitted tremendously from the cloud as building and training models became much easier as both storage and compute were are inputs into training ML models. Companies like Datarobot understood that building and deploying machine learning models required a combination of product, services, and top-down sales. The developer-led motion of AWS was not well-positioned to take on this opportunity and Datarobot was able to carve out room in a segment of the market. Eventually, AWS launched their well-known Sagemaker platform for ML, but that wasn’t until 2017, a decade after their entry into cloud computing.
Cybersecurity
Another area that created standalone businesses was on the security side. In part, this is because developers and security teams were completely separate to begin with. So while the cloud introduced new security problems, it was security teams that purchased solutions which was not AWS’ core sales muscle. There’s been various iterations from the initial CASB providers like Netskope and Symantec to newer CSPM providers like Orca and Wiz. Today, security has shifted left, blurring the lines of ownership between security and developers. We might expect to see the cloud giants push further into security as a result.
Data
The last area where we saw tons of standalone value created was in the data layer. From MongoDB to Databricks to Confluent, a number of companies were able to build data applications that benefitted from the scalability that AWS provided. Many of these companies were built around open source communities which AWS could not just throw dollars at to cultivate. Additionally, AWS was always more of a platform play, allowing for new vendors to cement themselves as best-of-breed and play nicely with others in the stack.
insight 3: New cloud-native markets can be more challenging for the cloud provider to come after. Because these problems are usually for a different user or buyer, it presents an opening for new entrants.
Consolidation
While many of these are standalone, public companies, it’s also true that AWS developed an incredibly dense catalog of services related to the cloud as well. In general, AWS dominates the categories closest to their technology primitives, while startups had more success building in the surrounding areas.
The end game of the cloud infrastructure market appears to be consolidation. As the growth rate of its core cloud computing businesses slows down, it makes sense that AWS would look to M&A to layer on growth opportunities. In addition to their M&A efforts, they’ve played more aggressively in markets that they previously left alone. For example, they have decided to directly compete with Elastic, who manages the open source project, Elasticsearch. Elastic was not pleased. In this cloud end game, I suspect, AWS will exercise its scale and ultimately be more combative than the last decade.
Closing
The cloud computing market followed the arc of abstractions → workflows → consolidation. I think there’s likely some pattern that will repeat itself there, but the more important takeaway for me is that when there’s a powerful infrastructure shift, there will be new problems and new companies to be built. Startups will always have speed and focus on their side providing a window of opportunity to compete even alongside large incumbents.
In the next post, I’ll dig in to what I think this could look like for LLMs and this next wave of AI innovation.
Feel free to reach me at @nanduanilal or email at nanduanilal@gmail.com if you have any thoughts on this post or are building something in the space.
I’m going to generally refer to AWS throughout, but much of this holds true for Google Cloud and Microsoft Azure as well
Microsoft would eventually acquire GitHub in 2018