Boosting Kubernetes Policy Compliance: Harnessing Kyverno Webhooks for Enhanced Validation

Our customer, a growing e-commerce company, faced challenges in maintaining consistent resource allocation and efficient usage across their Kubernetes cluster. With a complex microservices architecture and multiple teams managing deployments, they experienced occasional application failures, resource conflicts, and performance issues—mainly due to inconsistent container resource requests and limits. Manual monitoring and enforcement of these configurations became increasingly time-consuming and error-prone.

As part of our cloud consulting services and cloud security services, CloudifyOps recommended the implementation of Kyverno—a Kubernetes-native policy engine—to enforce best practices in resource management. We created a Kyverno policy titled “Require pod requests limits,” ensuring that all containers declared appropriate CPU and memory requests, along with upper consumption limits.

This automated policy enforcement helped the client prevent misconfigurations, reduce performance bottlenecks, and enhance governance across their Kubernetes environment. Kyverno, being an open-source policy engine, integrates seamlessly with Kubernetes and can validate configurations across deployments, pods, namespaces, and more.

Through our expert cloud consulting services and robust cloud security services, we enabled our customer to enforce governance at scale, minimize human error, and ensure consistent, secure, and optimized resource usage throughout their infrastructure.

How does Kyverno work?

Kyverno works by applying policies to the Kubernetes resources through the admission control mechanism. When a resource is created or modified, Kyverno’s webhook endpoints are invoked to intercept and process the requests. Kyverno evaluates policies based on resource information like metadata, labels, annotations, and namespace. Policies are defined with rules containing conditions to be evaluated against the Kubernetes resource. Each rule can have a validate, mutate, generate, or verifyImages declaration.

Types of Kyverno Policy

Policies in Kyverno can be either cluster-wide (ClusterPolicy) or namespaced (Policy). Namespaced policies only apply to resources within the same namespace, and cluster-wide policies are applied to matching resources across all namespaces. Other than this, there is no difference between the two types.

Installing the Kyverno Helm Chart in your Kubernetes Cluster

# Add the Kyverno Helm repository to your repository list helm
repo add kyverno https://kyverno.github.io/kyverno/

# Scan your Helm repositories to fetch the latest available charts
helm repo update

# Install the Kyverno Helm chart into a new namespace called “kyverno”
helm install kyverno kyverno/kyverno -n kyverno — create-namespace

A few of the commonly used policies and sample use cases:

Example Policies:

Here is a sample policy sample.yaml, which will restrict the kubernetes resource to be created, when we give the image tag `latest`.

apiVersion: kyverno.io/v1
kind: Policy
metadata:
name: disallow-latest-tag
namespace: demo
annotations:
policies.kyverno.io/title: Disallow Latest Tag
policies.kyverno.io/category: EKS Best Practices
policies.kyverno.io/severity: high
policies.kyverno.io/subject: Pod
policies.kyverno.io/description: >-
The ‘:latest’ tag is mutable and can lead to unexpected errors if the
image changes. A best practice is to use an immutable tag that maps to
a specific version of an application Pod. This policy validates that the image
specifies a tag and that it is not called `latest`.
spec:
validationFailureAction: audit
background: true
rules:
— name: require-image-tag
match:
resources:
kinds:
— Pod
validate:
message: “An image tag is required.”
pattern:
spec:
containers:
— image: “gcr.io/istio-testing/proxyv2 | *:*”
— name: validate-image-tag
match:
resources:
kinds:
— Pod
validate:
message: “Using a mutable image tag e.g. ‘latest’ is not allowed.”
pattern:
spec:
containers:
— image: gcr.io/istio-testing/proxyv2:latest | !*:latest”

Apply the policy in the Kubernetes cluster.

Kubectl apply -f sample.yaml

After applying the Kyverno policy, create a deployment in the Kubernetes cluster which has the `latest` image tag.

kubectl create deployment nginx — image=nginx:latest

Here, we can see the Kyverno policy is restricting the Kubernetes deployment from getting created, as we have mentioned the rules in the policy.

Now we can create the deployment again with a known registry.

kubectl create deployment nginx — image=nginx:1.25.0

Benefits achieved:

Improved Resource Efficiency — 20% decrease in resource consumption: The customer noticed a big improvement in using resources more efficiently when they enforced resource requests and limits. This resulted in better use of available resources and prevented wasting resources.

Enhanced Application Stability — 30% decrease in application failures and 15% reduction in incidents related to performance: The application’s stability improved because resources were consistently allocated according to set limits. This had a positive effect on the customer’s experience and minimized the disruptions caused by application downtime.

Cost Savings — estimated 15% cost reduction: The customer achieved cost savings by using Kyverno’s policy enforcement to optimize resource allocation and avoiding unnecessary overprovisioning.

Time and Effort Savings — 40% reduction: Automating the enforcement of resource allocation policies with Kyverno significantly reduced the manual effort required for monitoring and enforcing resource requirements. The customer’s DevOps teams could focus on strategic initiatives and improving development velocity.

Compliance and Governance: Kyverno offered the customer a centralized and consistent method to enforce resource allocation policies throughout their Kubernetes cluster. This assisted them in meeting compliance requirements and maintaining a governance framework, ensuring they followed the best practices for resource allocation.

Overall, Kyverno policies provide a powerful mechanism for managing and enforcing policies within Kubernetes, promoting cluster stability, security, and compliance with organizational standards.

To know more about how the CloudifyOps team can help you optimize your resource utilization, write to us today at sales@cloudifyops.com