Kubernetes Deployment from Platform to API | Yuriy Bezsonov | Conf42 Platform Eng. 2024
Table of contents
Speed up your app deployment by integrating runtime and backend services seamlessly on Kubernetes.
My name is Yuri Baso, and I am a Senior Solution Architect at AWS. In my role, I work with customers and AWS Partners on solutions related to container technologies. In today's talk, I'm going to walk you through a journey that we see customers facing when choosing to build their own internal development platform or internal delivery platform and expose internal APIs on top of Kubernetes.
We will discuss the different application delivery streams we see out there, what infrastructure controllers are, and how you can use them to accelerate the delivery time of your application. We will also define what compositions in Kubernetes are and the tools that support them. We will finish with an overview of a solution that you can try in your own environment.
Before we talk about delivery streams, let's define a problem statement and set up the context with a sample application. Our sample application is taken from the EKS workshop and is a good representation of an application that consists of multiple sub-applications, each with one or more components. These components can be runtime components, such as Kubernetes pods, deployments, services, and ingresses, which live inside the Kubernetes cluster. Additionally, there are infrastructure resources, such as AWS services, which live outside the cluster, for example, Amazon DynamoDB, Amazon MQ, or Amazon Relational Database Services.
When deploying an application, we need to consider not only deploying its runtime but also its backend services. This means taking care of components that live both inside and outside the Kubernetes cluster. In the next slides, I will walk you through some of the delivery streams we see out there, some of the delivery methods, and the time it takes to deploy an application in each stream.
The first stream is when developers simply deploy their runtimes and configurations to Kubernetes. Usually, the components, Kubernetes configuration, and its runtime are grouped together in a single repository. Kubernetes supports the creation and management of external networking and storage services from the very beginning. Those AWS resources corresponding to these services can be managed in a single stream as part of the application runtime configuration. For example, Kubernetes manifests such as services, ingress, and persistent volumes can, in turn, deploy application load balancers, network load balancing, and persistent load balancers using the Cetus constructs. This delivery stream usually takes minutes to complete.
The second, more complicated stream usually happens when additional backend services are needed. A pattern we see is an infrastructure definition repository that holds the entire organizational infrastructure as a code definition, for example, Amazon DynamoDB, RDS, S3, and other components. An infrastructure development or platform team may have different timelines, as these teams need to approve the pull request, which will trigger the creation of AWS services in the AWS account. Since this stream includes a handover process from one team to another, these delivery streams can take longer.
The last stream is the slowest one and involves some traditional ticketing system to provision resources. The database team here is just an example; it can be any other team. The pattern here is that developers have no way to influence or trigger a change in the infrastructure service and have to wait for another team to process their request to fulfill their tickets.
The question we ask ourselves, and developers often ask, is how we can make the deployment process faster. How can we improve the turnaround from changing a line of code in the application to deploying infrastructure and running the application in the cloud? Before diving into techniques and practices for achieving this improvement, let's set the requirements for each team because they are different.
From a platform team's perspective, they want to build a consistent deployment process while enforcing organizational standards. They need a secure environment, a compliant environment, and observability because they may have many environments from various teams. Application developers, on the other hand, want to have full ownership of their application components, including their backend infrastructure services, and they would like to have it in an easy way.
Streamline your code and infrastructure in the same repo to cut deployment time from hours to minutes.
The turnaround from changing the line of code of the application to deployed infrastructure and running application in the cloud involves several steps. Before diving into techniques and practices for achieving this and improving the process, it's essential to set the requirements for each team, as they differ significantly.
From a platform team perspective, the goal is to build a consistent deployment process while enforcing organizational standards. They need a secure and compliant environment, along with observability, because they may manage many environments across various teams. Application developers, on the other hand, want to have full ownership of their application components, including its backend infrastructure services. They desire an easy and fast process.
In the infrastructure as a code domain, there is a clear definition of boundaries and ownership when provisioning resources. According to the AWS Cloud Deployment Kit (CDK) best practices guide, the developer who manages the runtime should also manage the infrastructure. This is why infrastructure and runtime code should live in the same package and repository. By scanning the provided QR code, one can access the AWS CDK best practices guide for more details on this approach.
By applying this practice, developers manage both their runtime and infrastructure definition in the same code repository. Once the code is merged, two streams are triggered automatically: one to Kubernetes for runtime deployment, and another to infrastructure as a code tooling like AWS CDK or Terraform. These tools create the needed infrastructure, reducing deployment time from hours to minutes.
Focusing on Kubernetes, a similar process can be built using the well-known operators framework to build and maintain external services from within Kubernetes. Although writing your own operator is complex and involves undifferentiated heavy lifting, pre-built infrastructure controllers can significantly speed up implementation time. These controllers allow you to manage cloud services using your known Kubernetes API, creating a declarative infrastructure configuration. This helps leverage the Kubernetes ecosystem to provision both external and internal resources in the cluster.
Two main tools available now are AWS Controller for Kubernetes (ACK) and Crossplane. ACK, developed by AWS, enables the management of various AWS resources and services and is supported by AWS. Crossplane, a CNCF open-source project, has an extensible framework and a broad community around it.
Returning to the previous diagram, these tools help provision application backend services. Developers might worry about needing extensive knowledge of infrastructure configuration and deploying various cloud services. However, it's not necessary. For example, developers can simply request database instances without needing to know the specifics. They are responsible for configuring it upon creation, ensuring it runs, and modifying it when necessary. The platform team can decide on different implementations depending on whether the instance runs in the cloud or on-premise, abstracting the implementation details from the developer. For instance, the platform team might implement a database instance on-premise using a PostgreSQL database and provide instances for observability, while on AWS, they might choose a different setup.
Developers can focus on building apps while platform teams handle the infrastructure details.
Let's have a look at this example. At the bottom, developers simply request database instances. It doesn't really matter which instances they are; developers need a database. Of course, after that, they have the responsibility to configure it upon creation, ensure it is running, and be able to modify it when necessary to have control over that database instance. However, the platform team might decide to have different implementations depending on whether this instance runs in the cloud or on-premise and abstract the implementation details from the developer.
In our example, we see that the platform team can decide to implement this database instance on-premise using a PostgreSQL database and provide an instance for observability. On the other hand, on AWS, they want to leverage IAM for authentication, Amazon RDS for the database, and Amazon CloudWatch for observability, using more managed services to reduce operational overhead. This method is usually called composition, which means an abstraction of the underlying implementation details by providing a simple set of APIs.
Going back to infrastructure as code, we already have this capability of building abstractions of different implementations while providing clear interfaces on how to consume them. As you can see in this slide, these quotes can be taken from the documentation of AWS CDK and HashiCorp Terraform and describe exactly that composition is a key pattern for defining high-level abstractions through constructs. In AWS CDK, we even call it the same way: a composition.
Going back to the previous diagram, this is now how we would use infrastructure as code tooling with composition to abstract the implementation detail of our database instance. But how does composition look in Kubernetes? The CNCF Application Delivery Working Group published the platform white paper calling out two open-source projects available now that can help with compositions. Those projects are KubeVela and Crossplane.
KubeVela is a software delivery platform that implements an open application model. This model focuses on building applications rather than configuring Kubernetes objects and can be more natural and native for developers. Crossplane is the same tool that we discussed previously, but this time it is about the capability to build platforms on top of Kubernetes as well as providing infrastructure controllers. Another advantage of using the Kubernetes API to provision AWS resources is that you can leverage the extensive ecosystem tooling to package, deploy, and validate your infrastructure definitions. It's the same way you would use it with runtime configuration with your manifest deployment and pods. A small example can be using Helm, CDK8s for packaging components, ArgoCD for GitOps to reconcile the state of the repository with the state of the cluster, and Open Policy Agent for policy enforcement, along with a variety of other tooling from the Kubernetes ecosystem.
To demonstrate how to use it in practice, how to use composition and infrastructure controllers together, we at AWS built a solution reference for managing AWS services using infrastructure controllers through compositions. In this solution, we demonstrate how to provision an Amazon EKS cluster using compositions and infrastructure controllers. Yes, you can even provision the cluster itself with the help of compositions. Once a platform team adds a cluster composition manifest to the repository, the GitOps tooling reconciles the composition request, which in turn triggers the creation of a cluster with all the needed GitOps tooling deployed to that cluster. You have a cluster with all the necessary components, which allows you to do the next step.
The solution also describes an application onboarding process where a developer commits its runtime code and infrastructure configuration to the application-specific Git repository. After that, the developer creates a pull request to the centralized repository so the cluster GitOps tooling will start reconciling the application-specific repository. Once the platform team approves the PR, the GitOps tooling starts the reconciliation of the application repository. From there, the cluster infrastructure controller tooling will provision the relevant AWS resources for that specific application. It will create a complete lifecycle from the written line of code to the deployed application with all of the required infrastructure and solution components.
The solution itself is covered in a series of blog posts and has detailed sample code to get started with. If you scan this QR code, you will get additional information and be able to try this solution in your own AWS account. To summarize what we learned today, we saw how grouping runtime and infrastructure defines clear responsibility boundaries, how infrastructure controllers extend Kubernetes capabilities to provision AWS services. We also saw that the Kubernetes API can enable standardization, allowing us to leverage the Kubernetes ecosystem tooling, especially for provisioning not only Kubernetes resources but also infrastructure resources.
With that, I would like to say thank you. I hope you enjoyed this session, and please feel free to reach out on LinkedIn to have a discussion about anything related to platforms, containers, and architectures. I would really appreciate it if you could spend one minute answering a short survey about the session. Your feedback is very valuable. Thank you.