Kubernetes

• 1: K8S Request vs Limit
• 2: OAM 详解
• 3: Kubernates Cluster Architecture

1 - K8S Request vs Limit

不断有pod被重启，但是并不知道原因

名词解析

BestEffort: for a Pod to be given a Qos of BestEffort, the container in the Pod must not have any memory or CPU limits or requests.

Burstable: The Pod does not meet the criteria for QoS class Guaranteed, At least one Container in the Pod has a memory orr CPU request

Guaranteed: Every Container in the Pod must have CPU limit and a CPU request, and they must be same, Every Container in the Pod must have a memory limit and a memory request , and the must be the same.

要点

Request： 容器使用的最小资源要求，不限制容器的最大可使用资源。

Limit： 容器能使用资源的资源的最大值，设置为0表示使用资源无上限。

Memory目前只支持Request，Limit必须强制等于Request，这样确保容器不会因为内存的使用量超过了Request但没有超过Limit的情况下被意外的Kill掉

如果仅仅容器的内存使用量超过了Request但没有超过Limit的情况下，容器是不会被Kill掉的，所以上面这句话中”容器被意外的Kill掉“是其他原因导致的。比如出现内存资源抢占时。

Kubernetes 中资源的抢占

可压缩资源的抢占策略

— 按照Request的比值进行分配

不可压缩资源的抢占策略

按照优先级的不同，进行Pod的驱逐。

优先级？详细官网说明：Evicting end-user Pods

1. Request=Limit=0
2. 0 < Request < Limit < Infinity
3. 0 < Request==Limit

由于对不可压缩的资源，发生抢占的情况会出Pod被意外Kill掉的情况，所以建议对于不可以压缩的资源(Memory, Disk) 的设置为0<Requst==Limit

参考

1. Kubernetes 资源分配之Request和Limit解析 https://cloud.tencent.com/developer/article/1004976

2. Configure Quality of Service for Pods https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/

3. Kubernets 针对资源紧缺处理方式的配置（有点过时） https://www.kubernetes.org.cn/1150.html

2 - OAM 详解

名词解释

Actor Pattern (Actor 模型） 来自 Dapr 项目中的一个 concept. Dapr 项目也实现了 OAM 规范, Dapr 为未来 APP 开发方式。

Dapr is a progamming model and not an application mdoel, so it operates in a lower abstraction. OAM is intended to solve the problem of modeling an application and all dependencies.

Dapr will not model the application’s architecture.

它是一个并行计算的数学模型，最初为由大量独立的微处理器组成的高并行计算机所开发。Actor 模型的理念非常简单： 天下万物皆为 Actor。 Actor 之间通过发送消息来通信， 消息的传递是异步的。每个 Actor 是完全独立的，同时执行它们的操作。

当一个 actor 接收到消息后，它能做如下三件事情中一件：

• Create more actors
• Send messages to other actors
• Designates what to do with the next message

更多点击链接：10 分钟了解 Actor 模型，这本书也提到《七周七并发模型》，下载了电子版放在 MarginNote 3 里来细读。

Dapr 里实现了 Virtual Actor pattern^[2],

The Dapr actor runtime provides a simple turn-based Access model for accessing actor methods

背景

Application Developers – 编写程序实现业务价值，大部分不清楚 k8s 等其他基础设施。

Application Operators – 熟知集群的能力、稳定性、性能，以服务 Developers， 帮助 developer 配置、部署、操作应用（eg, updating, scaling, recovery)。 他们应该被称为 PaaS engineers。

Infra Operators – 服务 application operators, 应该被称为 IaaS 工程师。

OAM 的出现，解决了三者之间的合作问题 (Problems of Cooperation), 存在哪些常见的问题呢？

Interactions between Infra Operators and Application Operators

Application operator discover and manage capability that could potentially be in conflict with each other .

• DisCovery the spec of new capability is difficult
• Confirming the existence of specific capability in a particular cluster is difficult
• Conflicts in capability could be troublesome
  • Orthogonal
  • Composable
  • Conflicting

OAM’s Traits

In OAM, “Traits” are how we create capabilities with discorverability and manageability. 大部分的 traits 为 infra operator 来定义并实现的。

需要注意的是，Traits 并不等同于 K8s 插件， 一个集群有多个网络相关的 Traits， 比如“dynamic QoS trait”, “brandwidth control trait” 和 “traffic mirror trait”, 这些都是通过 CNI Plugin 来提供的。

Interactions between Application Operators and Application Developers

Developers' Voices Should be Heard

• several parameters for an application,
• Cannot be scaled
• Is a batch job, not a long running service
• Requires highest level security, etc

OAM’s Component

In OAM, we try to logically decouple K8s API objects, so developers can fill in their own intentions, and still be able to convey information to operatiors in structured manner.

The ApplicationConfiguration

ApplicationConfiguration 实例化应用，牵涉到 Component’s 名称和所应用到 Traits

合作流程的方式 ：

1. 基础平台提供不同的 workload Type
2. 开发者选择 workload type 定义 component.yaml
3. Application operator 执行 kubectl apply -f component.yaml 安装这个 component
4. Application operator 定义 ApplicationConfiguration 实例化这个应用。
5. 最后，Application Operator 执行 kubectl apply -f app-config.yaml 触发整个应用的部署。

OAM

A specification for describing applications as well as its operational capabilities,

In OAM, an Application is made from three core concepts,

1. The first is the Components, which might a collection of microservices, a database and a cloud load balancer
2. the second concept is a collection of Traits which describe the operational characteristics of the application such as capabilities like auto-scaling and ingress.
3. Finally, to transform these descriptions into a concrete application, operators use a configuration file to assemble components with corresponding traits to form a specific instance of an application.

Traits

represents a piece of add-on functionality that attaches to a component instance, such as traffic routing rules(including load balancing policy, network ingress routing, circuit breaking, rate limiting) , auto-scaling policies, upgrade strageies, and more .

Component

三部分组成：

• Workload 描述：怎么运行这个组件 (component)
• Component 描述： 运行的是什么东西
• 一组可重写的参数。

Workload Type 内置类型

如何选择内置的 workload type，根据三个方面来决定（如下）， 当然也可以扩展类型。

• Does this component expose service endpoint or not ?
• Is this component relicable or not
• Is this component long-runtime or one-time ( is daemonized or not )

Overwritable parameter

to operators, which part of my app definition is overridable

ApplicationConfiguration

Application Configuration 的一个简单例子如下：

apiVersion: core.oam.dev/v1alpha1
kind: ApplicationConfiguration
metadata:
  name: my-awesome-app
spec:
  components:
    - componentName: nginx
      instanceName: web-front-end
      parameterValues:
        - name: connections
          value: 4096
      traits:
        - name: auto-scaler
          properties:
            minimum: 3
            maximum: 10
        - name: security-policy
          properties:
            allowPrivilegeEscalation: false

其中

• parameterValues – Operator 使用这个字段来复写connections的值为 4096
• Trait auto-scaler – operator apply autoscaler trait (e.g. HPA) to the component.
• Trait security-policy – operator 应用安全策略规则到 component

Note: operator 可以增加更多的 traits 到列表， 比如可以增加’Canary Deployment Trait' 使应用更新符合 canary rollout strategy.

Scopes

Application scopes are used to logically group components together by providing application boundaries that represent common group behaviors and/or dependencies.

rudr scopes

Rudr 实践

缺国外云上的 k8s 环境，用于实践。 根据 rudr doc 实践

• helm 的使用
• rudr 目前的情况
• developer: 如何 Writing a Trait https://github.com/oam-dev/rudr/blob/master/docs/developer/writing_a_trait.md
• developer: how to implement your own workload type: support openfaas && prometheums as first extended workload. https://github.com/oam-dev/rudr/pull/481

安装 minikube (deprecated)

minikube start 下载 minikube.iso 文件速度太慢了，5kb/s

minikube-v1.6.0.iso: 19.92 MiB / 150.93 MiB 13.20% 9.44 KiB p/s ETA 3h56

放弃了。 使用 vultr 上 2c4m 机器来跑

在 vultr 上安装 microk8s

之前没有使用过 microk8s。也可以使用远程使用公司的电脑。

创建 ubuntu 18.04 系统，使用 microk8s，使用 helm3 安装 rudr

参考 Installing rudr。

helm3 Github Homepage: https://github.com/helm/helm Helm is a tool for managing Charts. Charts are packages of pre-configured kubernetes resources. Think of it like apt/yum/homebrew for Kubernetes.

Charts can be stored on disk, or fetched from remote chart repositories(like Debian or RedHat packages)

microk8s Homepage: https://github.com/ubuntu/microk8s The smallest, fastest Kubernetes. Single-package fully conformant lightweight kubernetes Perfect for:

• Developer workstations
• IoT
• Edge
• CI/CD

snap install microk8s --classic

microk8s.enable helm3

git clone https://github.com/oam-dev/rudr.git

microk8s.helm3 install rudr ./rudr/charts/rudr --wait

Installing Implementations for Traits

Ingress，Manual Scaler, autoscaler 比如 Ingress，那么从 Helm3 hub 中选择 Ingress 的实现即可。 同样， autoscaler 选择一种实现 并部署即可。

Deploy a sample Rudr application using the tutorial

教程： Tutorial: Deploy, inspect, and update a Rudr application and its components

比较简单的完成。 心得： 如果 Traits 比较丰富并且能轻松获取到使用文档的话，对于 Application Operator 来说，是非常方便的对 Application 的加上增强或者限制的功能。

OAM 的未来

working with the community on OAM spec as well as K8s implementation

参考

[1] Phil Bernstein Sergey Bykov Alan Geller Gabriel Kliot Jorgen Thelin: Orleans: Distributed Virtual Actors for Programmability and Scalability

[2] Dapr: Actors overview: Overview of the actors building block

[3] The Open Application Model from Alibaba’s Perspective https://www.infoq.com/articles/oam-alibaba/

[4] 开放应用模型 OAM https://www.jianshu.com/p/da9bf3357247

[5] A Kubernetes implementation of the Open Application Model specification https://github.com/oam-dev/rudr

[6] Automating Event-Based Continuous Delivery on Kubernetes with keptn, The keptn project provides a clear separation of concerns, allowing developer, DevOps and site reliability engineers to update delivery pipelines, 这个更加细，包括 SLO 指标、包括 blue\green deploy

3 - Kubernates Cluster Architecture

Cloud Controller Mananger

解耦 Kubernetes 和底层云原生基础的交互。让两者的更新以不同的速度进行。

Cloud-controller-manager 使用插件机制，让不同的 cloud providers 集成 Kubernetes 到他们的平台。

Cloud controller manager runs in the control plane as a relicated set of processes(usuallyh, these are containers in Pods). Each cloud-controller-manager implements multiple controllers in a single process.

The controllers inside the cloud controller manager include:

• Node Controller
• Route Controller
• Service Controller

cloud-provider-alibaba-cloud 使用阿里云负载均衡来访问服务。链路更短，

原链路： SLB –> Node –> Pod

新链路： SLB –> Pod

参考资料

IBM 微讲堂：Kubernetes 系列