“Platform engineering is not a DevOps killer; rather, it is an evolution of DevOps that enables true DevOps at scale.”
This was the main takeaway from Luca Galante, VP of Product & Growth with Humanitec, who shared his practical perspective on the current state of DevOps and the rise of platform engineering. Joined by Cloudsmith’s CEO Glenn Weinstein and Chief Strategy Officer Alan Carson, the technology veterans had a candid conversation about the disruption of DevOps during our recent webinar, From DevOps to Platform Engineering: The Next Evolution.
Here are the five key lessons I took away from their in-depth discussion:
1. Simply Put, DevOps Does Not Scale
The principles that led to the adoption of DevOps are still very much alive and relevant today. DevOps continues to work well in smaller teams where everyone is familiar with the toolchain.
However, challenges arise when scaling DevOps practices to larger teams with hundreds or thousands of developers and a complex toolchain. In such scenarios, expecting every team member, especially new hires, to be familiar with the entire toolchain and perform operations end-to-end is unrealistic, leading to bottlenecks and development delays.
The factors behind DevOps' scalability challenges include:
- Growth of Engineering Organizations: Engineering teams have expanded rapidly over the years. For instance, AWS started with around 300 engineers, but some companies have since experienced a hundredfold increase. This growth created a need for more efficient infrastructure and service management.
- Expansion of In-House Engineering in Enterprises: Over the past two decades, companies like McDonald's and Ford have significantly expanded their in-house engineering teams. What were once small developer groups have become substantial engineering departments, driving demand for platform engineering solutions.
- Onboarding Large Numbers of Developers: Companies like Spotify, Google, and Airbnb were among the first to recognize the limitations of traditional DevOps when scaling to onboard many developers. This realization led them to adopt platform engineering practices to streamline operations and collaboration.
Today, with hundreds of thousands of developers navigating complex cloud and hybrid cloud environments, platform engineering has become essential.
2. Platform Engineering Simplifies Cloud Complexity
The rapid expansion of the technology ecosystem, as evidenced by the Cloud Native Computing Foundation (CNCF) landscape with its thousands of tools, has introduced layers of complexity for developers. Adding to the intricacy of modern development environments, the significant cloud-native trends that have impacted the need to re-engineer the way DevOps approaches software development include:
- The Need for Containerization: The adoption of containerization technologies, such as Docker, significantly impacted the development landscape by allowing applications and their dependencies to be packaged together, providing consistency across different environments.
- The Rise of Kubernetes: Open-source container orchestration platforms like Kubernetes revolutionized how applications are deployed and managed at scale. However, Kubernetes also introduced new challenges and complexities in managing containerized applications.
- Infrastructure as Code (IaC): Adopting Infrastructure as Code (IaC) practices, where infrastructure configurations are managed through code, brought automation and consistency to infrastructure provisioning and management but also introduced control issues.
While these technologies can independently ease processes, their collective intricacies create new challenges for developers. Platform engineering addresses these issues by equipping developers with self-service platforms that effectively abstract and manage these complexities.
3. Platform Engineering Is the Key to Unlocking Developer Autonomy
Recognizing the intertwined nature of DevOps and platform engineering, the challenges of observability and efficiency in large-scale organizations demands standardization to unify distributed teams and fragmented tools.
Platform engineering addresses these challenges by creating an internal developer platform (IDP) that consolidates and streamlines tools and processes, offering a more scalable and integrated solution than traditional DevOps, including:
- Developer Self-Service: Platform engineering abstracts complexities by providing a self-service platform layer with “golden paths” that balance ease of use with necessary constraints. This empowers developers to independently access and manage their tools—reducing cognitive load, minimizing workflow delays, and freeing them from infrastructure management.
- Consolidation and Standardization: By unifying disparate tools and technologies into a cohesive platform, platform engineering enhances observability and integration across the organization. This addresses the inefficiencies and lack of visibility that can arise from managing multiple CI/CD tools, hundreds of language formats, and dispersed development environments.
- Scalability: Platform engineering delivers a scalable solution that evolves with the organization, overcoming the limitations of DevOps practices that struggle to support large teams. It automates processes and reduces the reliance on manual configurations, which often become bottlenecks in larger enterprises.
- Platform as a Product: The platform engineering team treats IDP as a product, designed to meet the specific needs of internal developers. This product-oriented approach shifts the focus from traditional infrastructure maintenance to providing a user-friendly platform that abstracts complexities and boosts productivity.
While developers may initially resist the “guardrails” and standardization that platform engineering introduces, they actually do get value with the freedom to focus on what they do best: software development. On the other hand, operations teams are quick to recognize the value of its streamlined workflows and security teams appreciate its automated policy enforcement.
4. Platform Engineering Adoption Requires Key Stakeholder Buy-In
Implementing clearly defined frameworks and processes and driving cultural transformation are crucial strategies for successfully rolling out a platform engineering initiative. Convincing stakeholders—such as developers, security, operations, architects, and executives—can be a significant challenge.
Galante observed that one common pitfall for organizations is attempting to roll out platform engineering to everyone at once. To gain buy-in across the organization, consider these strategies:
- Start with Experienced Teams: Begin with teams open to experimenting with new technologies, such as application developers, and gradually expand. Success with early adopters can create a positive ripple effect, encouraging wider adoption.
- Drive Cultural Transformation: Engage stakeholders, understand their interests, and tailor messaging to address their specific concerns. Align the platform's benefits with individual and organizational goals to increase the likelihood of adoption.
- Highlight Immediate Benefits: Ensure the platform is user-friendly and enhances efficiency. Emphasize benefits like faster deployment times, reduced manual tasks, and improved productivity to motivate developers to embrace the new platform. Demonstrating these advantages can accelerate adoption.
With these steps in mind, you’ll be well on your way to creating a more scalable, integrated, and developer-friendly environment while overcoming traditional DevOps limitations.
5. Platform Engineering Is Empowering Software Development
The primary role of a platform engineering team is to provide an internal developer platform that makes decisions on behalf of developers, allowing them to focus on their areas of expertise without needing to be generalists. It is about driving standardization by design across your entire engineering setup.
While many resist leaving behind their homegrown or legacy platforms, the benefits for large-scale organizations make the choice clear. With platform engineering in place, you will:
- Free Developers: Enable developer self-service to reduce the cognitive load on developers and prevent them from being stuck waiting for resources.
- Streamline Operations: Address the constant flow of operational tickets that burden ops teams, preventing them from becoming organizational bottlenecks.
- Secure Your Software: Minimize manual configurations and ad hoc solutions to improve scalability within the organization.
Galante, who also leads PlatformCon, estimates that 20-30% of large software organizations with around a hundred developers already have a platform engineering team or practice—a growing trend. Gartner predicts this adoption rate will reach 80% within two years, driving significant organizational transformations.* As platform engineering becomes more widespread, these organizations will see substantial improvements in processes, productivity, and security.
More Platform Engineering Content
Webinars
If you are considering platform engineering, watch the complete webinar (part one of a three-part series on DevOps Disruptors). You’ll get to hear directly from the experts who do a deeper dive into how IDPs can promote greater agility and innovation and the critical success factors you need to know before you transition.
Free report
Make sure to also grab your free copy of Omdia's industry report Market Disruptors: Moving to Platform Engineering Does Not Mean the End of DevOps for advice on planning your platform engineering strategy, benchmarking data, and insights on key tools helping teams to accelerate their transition.
*Source: Gartner’s Top Technology Trends for 2024