Enhancing Kubernetes Security and Compliance with Kyverno

Amit Sharma
5 min readJun 24, 2024

Introduction

Kyverno is a Kubernetes native policy management tool designed to streamline the process of validating, mutating, and generating Kubernetes configurations. As organizations increasingly adopt Kubernetes, ensuring compliance, security, and operational consistency becomes crucial. Kyverno provides an intuitive and flexible approach to policy enforcement using Kubernetes Custom Resource Definitions (CRDs). This article will guide you through the steps to implement Kyverno in a Kubernetes cluster, focusing on installation, configuration, and practical use cases.

What is Kyverno?

Kyverno, which means “govern” in Greek, allows Kubernetes users to define and enforce policies using Kubernetes CRDs. Unlike traditional policy engines that require learning new languages, Kyverno policies are defined in YAML, making them accessible to Kubernetes administrators and developers. Key functionalities of Kyverno include:

  • Validation: Ensuring resources meet specified criteria.
  • Mutation: Automatically modifying resources to meet desired configurations.
  • Generation: Creating new resources in response to changes in existing resources.

Prerequisites

Before implementing Kyverno, ensure you have the following:

  1. A Kubernetes cluster (v1.16 or newer).
  2. kubectl command-line tool configured to interact with your cluster.

Prerequisites

Before implementing Kyverno, ensure you have the following:

  1. A Kubernetes cluster (v1.16 or newer).
  2. kubectl command-line tool configured to interact with your cluster.

Installing Kyverno

Kyverno can be installed using Helm, a Kubernetes package manager, or directly applying the YAML manifests. Below, we demonstrate both methods.

Using Helm

  1. Add the Kyverno Helm repository:
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update

2. Install Kyverno in the kyverno namespace:

kubectl create namespace kyverno
helm install kyverno kyverno/kyverno -n kyverno

Using YAML Manifests

  1. Apply the Kyverno installation manifest:
kubectl apply -f https://raw.githubusercontent.com/kyverno/kyverno/main/definitions/install.yaml

Verify the Kyverno components are running:

kubectl get pods -n kyverno
NAME                                                       READY   STATUS      RESTARTS   AGE
kyverno-admission-controller-f789cf5f-xfcm2 1/1 Running 0 48m
kyverno-background-controller-8977d8fd9-46qvv 1/1 Running 0 48m
kyverno-reports-controller-56b7bbbd7d-tfwqc 1/1 Running 0 48m

Creating and Applying Policies

Kyverno policies are written as Kubernetes resources. They can validate, mutate, or generate other resources based on predefined conditions. Below are examples of each type of policy. Here, i will showing you two of them as mentioned and described below.

1. Validation Policy

This policy ensures that every pod must have a specific label. The below kyverno policy targets pods, secrets, service and configmap. Here we targetting on a pod resource.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
annotations:
policies.kyverno.io/title: Require Labels
policies.kyverno.io/category: Best Practices
policies.kyverno.io/minversion: 1.6.0
policies.kyverno.io/severity: medium
policies.kyverno.io/subject: Pod, Label
policies.kyverno.io/description: >-
Define and use labels that identify semantic attributes of your application or Deployment.
A common set of labels allows tools to work collaboratively, describing objects in a common manner that
all tools can understand. The recommended labels describe applications in a way that can be
queried. This policy validates that the label `app.kubernetes.io/name` is specified with some value.
spec:
validationFailureAction: audit
background: true
rules:
- name: check-for-labels
match:
any:
- resources:
kinds:
- Pod
validate:
message: "The label `app.kubernetes.io/name` is required."
pattern:
metadata:
labels:
app.kubernetes.io/name: "my-app"

Apply the policy:

kubectl apply -f require-pod-labels.yaml

Now if you try to create any pod without labels it will give you warnings in the events.

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning PolicyViolation <unknown> policy require-labels/check-for-labels fail: validation error: The label `app.kubernetes.io/name` is required. rule check-for-labels failed at path /metadata/labels/app.kubernetes.io/name/

2. Mutation Policy

This policy automatically adds a label to pods if it does not exist:

apiVersion : kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-default-resources
annotations:
policies.kyverno.io/title: Add Default Resources
policies.kyverno.io/category: Other
policies.kyverno.io/severity: medium
kyverno.io/kyverno-version: 1.10.0-alpha.2
policies.kyverno.io/minversion: 1.7.0
kyverno.io/kubernetes-version: "1.26"
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
Pods which don't specify at least resource requests are assigned a QoS class
of BestEffort which can hog resources for other Pods on Nodes. At a minimum,
all Pods should specify resource requests in order to be labeled as the QoS
class Burstable. This sample mutates any container in a Pod which doesn't
specify memory or cpu requests to apply some sane defaults.
spec:
background: false
rules:
- name: add-default-requests
match:
any:
- resources:
kinds:
- Pod
preconditions:
any:
- key: "{{request.operation || 'BACKGROUND'}}"
operator: AnyIn
value:
- CREATE
- UPDATE
mutate:
foreach:
- list: "request.object.spec.[ephemeralContainers, initContainers, containers][]"
patchStrategicMerge:
spec:
containers:
- (name): "{{element.name}}"
resources:
limits:
+(memory): "100Mi"
+(cpu): "100m"
requests:
+(memory): "100Mi"
+(cpu): "100m"

Now when you create a pod without passing limits and requests it will take automatically. Below you can see the output of the pod resource which has limit and Requests already included.

    State:          Running
Started: Sun, 23 Jun 2024 14:42:51 +0000
Ready: True
Restart Count: 0
Limits:
cpu: 100m
memory: 100Mi
Requests:
cpu: 100m
memory: 100Mi
Environment: <none>

Similary, you can create multiple mutation for QOS classes.

In Kubernetes, Quality of Service (QoS) classes are used to manage the allocation of resources (CPU and memory) to Pods based on their resource requirements and limits. These classes help Kubernetes prioritize and allocate resources more effectively, especially during periods of resource contention. There are three QoS classes in Kubernetes: Guaranteed, Burstable, and BestEffort. Each class is determined by the resource requests and limits specified for the containers in a Pod.

  1. Guaranteed:
  • Description: Pods are assigned to the Guaranteed QoS class if all their containers have memory and CPU limits and requests set to the same values.
  • Characteristics:
  • These Pods are given the highest priority.
  • They are guaranteed the requested resources and are not subject to eviction under normal conditions.
Limits:
cpu: 100m
memory: 100Mi
Requests:
cpu: 100m
memory: 100Mi

2. Burstable:

  • Description: Pods are assigned to the Burstable QoS class if they have at least one container with a memory or CPU request that is less than the limit, or if some containers have requests and others do not.
  • Characteristics:
  • These Pods have a higher priority than BestEffort but lower than Guaranteed.
  • They can burst up to the specified limits if resources are available.
  • They are subject to eviction under memory pressure, but only after BestEffort Pods.
    Limits:
cpu: 100m
memory: 100Mi
Requests:
cpu: 80m
memory: 80Mi

3. BestEffort:

  • Description: Pods are assigned to the BestEffort QoS class if none of their containers have memory or CPU requests or limits set.
  • Characteristics:
  • These Pods have the lowest priority.
  • They do not have guaranteed resources and are the first to be evicted under resource pressure.

Conclusion

Implementing Kyverno in your Kubernetes cluster enhances security, compliance, and operational efficiency through easy-to-define policies. By leveraging Kyverno’s native Kubernetes integration, you can enforce best practices and maintain cluster integrity seamlessly. Start by installing Kyverno, create policies tailored to your needs, and monitor their enforcement to ensure your Kubernetes environments remain robust and compliant.

References:

--

--

Amit Sharma

4xGCP || 2xRedHat Certified || DevOps Engineer || Terraform || Ansible || GitLab || Jenkins || Kubernetes || Docker || Openshift || AWS || GCP || Azure