Kubernetes Implementation Design
Kubernetes is said to be composed of microservices which form the aforementioned three planes.
But an important distinction needs to be made: microservices manage and encapsulate their state with their own API protocol, whereas Kubernetes fully defines a common API framework and manages the state for all its components in the data plane
There is ample external documentation available, which describes the architecture and detailed implementation of Kubernetes (primarily as container orchestrator).
The authors of Kubernetes made early design choices which simplify the efficient bootstrapping and setup of a cluster from zero (for example, instead of introducing a message bus microservice - like many similar management systems - all communication between the components is handled via the API server, which reduces the setup complexity).
The same principles used in Kubernetes can be extended to serve in the reference architecture to integrate every type of capabilities, beyond container orchestration within a single cluster.
Scalability and fault tolerance in Kubernetes are implemented entirely in software with the assumption that anything can fail anytime. A typical setup for all planes therefore consists of multiple independent nodes (machines or virtual machines) where components scale and operate redundantly even in cases of failure.
Furthermore, Kubernetes provides a universal runtime since almost all workloads can be packaged into containers, including the described planes. As a point of proof, Kubernetes itself (as many other platforms with their own control
Kubernetes Planes
In the following, we will untangle the conceptual building blocks/planes from Kubernetes as interpreted in the reference architecture.
Components of the data plane
a) The persistency: Kubernetes and kcp
kcp
b) the API server (in the data plane, we assume the API server to have no use case specific opinion; cf. generic control plane).
The goal of the data plane is to provide c) controlled access to a shared repository for all digital twin
kcpkcp
Components of the Control Plane
If we conceptually attribute the API server to the data plane
a) all needed API Extensions.
The Kubernetes core design pattern includes b) scheduler, and
c) controller manager which bundles various Kubernetes internal controllers.
Furthermore, a typical cluster minimally requires d) cloud controller manager which bundles controllers necessary for dealing with resources from external cloud providers.
Administrators can add e) additional, but optional features and their controllers (for example VPA or API Gateways).
Kubernetes also requires f) a few controllers to run on each worker node (in the work plane) to bring the nodes under management. Conceptually, it can be argued that they belong to the control plane.
The data plane and control plane can be colocated, or they can run separated. Collectively they function as the orchestration brain. API requests are processed, converted into desired states, which are picked up by the various controllers, some of which translate the desire into imperative instructions that will move the actual state on the work plane. Scheduling is implemented entirely in the virtual representation, using available observability metrics reported back into the digital twins.
Managed Kubernetes-as-a-Service are often built with an abstracted Control-Plane-as-a-Service.
Components of the Work Plane
Its main purpose is to bring the capabilities (compute, network, and storage) of multiple worker nodes under management.
The work plane
b) essential actuators/controllers and drivers/plugins which actuate the capabilities on each of the nodes.
For compute, Kubernetes utilizes the kubelet controller which interacts with CRI and a fitting container runtime, such as containerd or CRI-O. For networking, the kube-proxy controller (or modern equivalent) and a CNI driver is needed. For storage, a specific CSI controller setup is expected. Kubernetes can be extended to manage further capabilities (for example GPUs, or virtualization) by deploying specialized extensions, controllers, and their drivers in the control and work plane.
These components are quasi part of the control plane, they pull the desired state from the data plane, compare with the local actual state and compute any necessary local instructions: create or destroy containers, manage the associated networking (which enable communication between containers in and outside the cluster), as well as handle externalized storage for the containers.
Connecting the Planes
With the foundational knowledge of the three planes at hand, we can connect them to form a cluster.
The controllers map the desired state described by the digital twins to the real world, while reporting back the actual state. The real world for a digital twin in this Kubernetes configuration consists of a work plane for containers, which can be used to implement any desired workload. The work plane can be constructed with manually attached machines, or with automation using an infrastructure provider and corresponding actuators for machines.
