Unlocking the Future of Kubernetes Policy as Code With Kyverno - Vishal Choudhary & Frank Jogeleit

Unlocking the Future of Kubernetes Policy as Code With Kyverno

Speakers: Frank Jogeleit, Senior Software Engineer at Nuvata; Vishal Choudhary, Kyverno Maintainer

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=L13y_-zLin4

Overview

In the rapidly evolving landscape of cloud-native security and governance, Kubernetes policy as code has become an indispensable tool for maintaining compliance and operational integrity. This talk by Frank Jogeleit and Vishal Choudhary at KubeCon EU delves into the significant evolution of Kyverno, a CNCF incubating project designed to bring native policy enforcement to Kubernetes. The presentation outlines Kyverno's current capabilities and, more importantly, unveils an ambitious roadmap centered around simplifying its API, standardizing on Common Expression Language (CEL), and extending its reach beyond Kubernetes resources to any JSON payload.

The core motivation behind these changes stems from a desire to align Kyverno more closely with Kubernetes' own direction for admission control, particularly its adoption of CEL for Validating Admission Policies (VAP). By embracing a more unified and expressive policy language, Kyverno aims to overcome the complexity of its existing multi-faceted API, offering a more intuitive and powerful experience for users. This strategic shift promises not only improved performance and a clearer policy definition but also a future where Kyverno can serve as a universal policy engine for cloud-native environments, facilitating advanced use cases like supply chain security and dynamic resource management across diverse infrastructure.

Background

▶ Watch: Introduction and talk agenda (0:00)

Kyverno, deriving its name from the Greek word "to govern," has established itself as a robust, Kubernetes-native policy engine. Unlike other policy solutions that often require learning a new programming language (like Rego for OPA), Kyverno's primary strength has been its YAML-native approach, allowing users to define policies directly using Kubernetes manifests. Its current feature set is extensive, encompassing several critical functions for Kubernetes governance:

• Validation: Kyverno acts as an admission controller, intercepting resource requests to ensure they adhere to predefined rules. Policies can use patterns, assertion trees, JamesPath, and basic CEL expressions to validate resource configurations, such as the presence of specific labels or correct CPU/memory limits. It also supports port security policies and can perform background scans on existing cluster resources to report on compliance status, generating policy reports for auditing.
• Mutation: Policies can modify resources before they are admitted to the cluster, using strategic merge patch or JSON patch. This allows for automatic injection of security contexts, labels, or other configurations, ensuring resources meet requirements without blocking them outright. Kyverno can also mutate existing resources.
• Generation: This feature enables Kyverno to automatically create new resources based on conditions. A common use case is generating default network policies for new namespaces or creating Vertical Pod Autoscalers (VPAs) for StatefulSets and Deployments. A notable example cited was Adidas using this to reduce cluster costs by 50% by automating VPA creation. Generated resources can be synchronized with a source object, providing tamper resistance.
• Cleanup: Kyverno can periodically identify and delete resources that violate conditions or have expired. This is useful for removing unused resources, cleaning up orphaned Pod Disruption Budgets (PDBs) that might block node shutdowns, or setting Time-To-Live (TTL) on resources.
• Image Verification: Crucial for supply chain security, Kyverno can verify container images based on signatures from solutions like Notary and Cosign (including keyless and certificate-based signing). It also supports verifying signatures on attestations (e.g., SBOMs or vulnerability scan reports), allowing policies to check the payload of these attestations for specific conditions, such as the absence of critical vulnerabilities or adherence to specific SBOM formats. Kyverno recently added support for GitHub artifact attestation.

Despite this rich feature set, the existing architecture presented challenges. Kyverno's multiple ways to achieve similar logic (patterns, JamesPath, basic CEL) led to a large and complex API, making it difficult for users to choose the "best" approach and for maintainers to provide consistent support. Simultaneously, Kubernetes itself has moved towards standardizing admission policies with its Validating Admission Policy (VAP) feature, which exclusively uses CEL for expression evaluation. This created a natural impetus for Kyverno to realign, simplify, and expand its capabilities while maintaining its core philosophy of being Kubernetes-native and user-friendly.

Key Findings

▶ Watch: Kyverno's core capabilities: admission control, scanning, auditing (2:00)

The central finding and direction of this talk is Kyverno's strategic evolution to streamline its policy definition, enhance expressiveness, and broaden its applicability. This transformation is driven by several key initiatives:

1. API Simplification and New CRDs: Kyverno is moving away from a single, monolithic ClusterPolicy CRD that encapsulated all rule types. Instead, it is introducing a suite of specialized Custom Resource Definitions (CRDs): ValidatingPolicy, ImageValidatingPolicy, MutatingPolicy, GeneratingPolicy, and CleanupPolicy. This modularization makes the API clearer, more focused, and easier to understand, with ValidatingPolicy and ImageValidatingPolicy being the first to arrive in Kyverno 1.14.

1. Standardization on CEL: Recognizing the industry trend and Kubernetes' own adoption of CEL for native VAPs, Kyverno is standardizing on CEL as its primary expression language. This provides a powerful, side-effect-free, and widely supported language with a rich feature set, simplifying policy authoring and fostering greater interoperability.

1. Extension of CEL with Custom Libraries: While embracing CEL, Kyverno is committed to retaining its unique, advanced features. This is achieved by extending the core CEL implementation with custom libraries that provide functionalities not natively available in Kubernetes VAPs, such as resource lookups, external service calls (http.get), and ConfigMap lookups. This ensures that users can transition to the new API without losing existing capabilities.

1. Support for Any JSON Payload: A significant expansion is Kyverno's ability to apply policies to any JSON-based payload, not just Kubernetes resources. By setting an evaluationMode: JSON, users can leverage the same powerful CEL-based policies to govern external configurations, CI/CD artifacts, or other JSON data, positioning Kyverno as a more versatile governance tool across the cloud-native ecosystem.

1. Enhanced Image Verification: The ImageValidatingPolicy CRD introduces specialized constructs for robust supply chain security. This includes an images variable for identifying image locations in diverse JSON structures, dedicated attesters for integrating with signing solutions like Notary and Cosign, and specific CEL functions like verifyImageSignature and verifyAttestationSignature. Critically, it allows for granular validation of attestation payloads, enabling checks against SBOM formats (e.g., CycloneDX) or vulnerability reports.

These findings collectively represent a significant leap forward for Kyverno, making it more aligned with the Kubernetes ecosystem, more powerful through CEL extensions, and more flexible by supporting non-Kubernetes JSON payloads.

Technical Deep Dive

▶ Watch: Kyverno's Validation policies for compliance checks (3:40)

Kyverno's evolution centers on addressing the complexity of its existing API and aligning with Kubernetes' adoption of CEL. Previously, Kyverno offered a single ClusterPolicy CRD with various rule types (validate, mutate, generate, cleanup, verifyImages) and multiple expression languages (patterns, JamesPath, basic CEL) within each. This "really large API" made it difficult for users to determine the optimal solution and for maintainers to support.

The decision to standardize on CEL (Common Expression Language) is strategic. CEL offers a comprehensive feature set comparable to JamesPath but with a larger community and ecosystem, providing a powerful, side-effect-free language for expression evaluation. Kyverno's goal is to be as cloud-native as possible, and by leveraging the standardized Kubernetes API for Validating Admission Policies (VAP) as a base, it aims to simplify the transition for users and improve performance.

The new design introduces five distinct policy types, each with its own CRD, clarifying policy boundaries:

• ValidatingPolicy
• ImageValidatingPolicy
• MutatingPolicy
• GeneratingPolicy
• CleanupPolicy

Of these, ValidatingPolicy and ImageValidatingPolicy are the first to be released in Kyverno 1.14 (RC builds were available at the time of the talk).

ValidatingPolicy

The ValidatingPolicy CRD is built upon the Kubernetes VAP API, but with crucial extensions that overcome VAP's inherent limitations (e.g., time constraints preventing external API calls). Key Kyverno-specific additions include:

• evaluationConfig: This block allows users to configure background scanning (backgroundEnabled: true) and admission disabling, replicating existing Kyverno features.
• auditAnnotation: This enables the addition of custom properties to policy reports, useful for compliance and detailed auditing.
• Extended CEL Libraries: Kyverno extends CEL with custom functions and libraries to support advanced use cases:
• Resource Lookups: Policies can now access other resources within the cluster (e.g., fetching a ConfigMap).
• External Service Calls: The http.get library allows policies to make API calls to external services, enabling dynamic validation based on external data sources.
• ConfigMap Lookups: Specific functions for retrieving and parsing data from ConfigMaps.

A significant innovation is the support for any JSON payload. By setting evaluationMode: JSON, the ValidatingPolicy can be applied to arbitrary JSON structures (e.g., Dockerfiles, SBOMs, custom configuration files). This extends Kyverno's governance capabilities beyond Kubernetes, allowing it to validate JSON data in CI/CD pipelines or other contexts using the same policy engine. For instance, a policy could verify a parsed Dockerfile's JSON representation to prohibit the use of curl commands.

ImageValidatingPolicy

The ImageValidatingPolicy CRD is also based on the VAP API but is specialized for image verification and supply chain security. It introduces dedicated fields and functions to handle the complexities of image signing and attestation:

• images Variable: For generic JSON payloads, users can specify the location of image references using a CEL expression (e.g., images: "$.containers[*].image"). For standard Pods or Pod controllers, this is autofilled. This allows Kyverno to extract images from diverse resource types for verification.
• attesters Block: This defines trusted authorities for image and attestation verification. It supports:
• Notary: Verification using Notary certificates.
• Cosign: Comprehensive support for Cosign features, including keyless signing, KMS integration, and certificate-based verification.
• CEL Functions for Verification:
• verifyImageSignature(image, attesters): This function takes an image reference and a list of defined attesters and returns a count of successful verifications. Policies can then assert that count > 0 to ensure an image is signed by at least one trusted attester.
• verifyAttestationSignature(image, attestation, attesters): This function is used to verify signatures on attestations linked to an image (e.g., SBOMs, vulnerability scan reports). It supports OCI referrers API attestation and can check the authenticity of attestations like SPDX or CycloneDX SBOMs.
• Attestation Payload Verification: Beyond signature verification, policies can inspect the payload of an attestation. For example, a policy can fetch the SBOM payload and use CEL expressions to ensure its format is cycloneDX or to check for the absence of critical or high-severity vulnerabilities reported in a vulnerability scan attestation. This provides fine-grained control over the security posture of deployed images.

This new architecture strikes a better balance between declarative policy definition and imperative syntax through CEL, offering a simpler, more expressive API while retaining and expanding Kyverno's powerful feature set.

Demo / Proof of Concept

▶ Watch: Kyverno's Generation policies for creating new resources (6:00)

The speakers provided a live demonstration using a Kyverno playground environment, featuring a preview build of Kyverno 1.14 with the new policy types. A QR code was provided for attendees to access a GitHub repository containing all demo resources and links to the playground.

ValidatingPolicy Demonstrations

1. Basic Deployment Label Check:

• A ValidatingPolicy was shown targeting Deployment resources.
• It used a match condition to target deployments and a variable block to extract a label (e.g., app.kubernetes.io/name) from the resource.
• The validation rule then checked if the extracted label's value was "prod" using a simple CEL expression (variable.labelValue == 'prod').
• Two example resources were tested: a "good deployment" with the correct label, which passed, and a "bad deployment" with an incorrect label, which failed with a user-defined error message.
• The demo highlighted that Kyverno automatically generates rules for related controllers (ReplicaSets, StatefulSets) for Pod-level policies.

1. Resource Lookup (ConfigMap-based Namespace Whitelist):

• This demo illustrated Kyverno's extended CEL capabilities for resource lookups.
• A ConfigMap named allowed-namespaces in the default namespace contained a comma-separated list of allowed namespaces (e.g., "dev,test,stage").
• The ValidatingPolicy used a variable to fetch this ConfigMap's data.
• The validation rule then converted the ConfigMap string into a list (configMapData.split(',')) and checked if the object.namespace was in this list.
• An example resource in the "dev" namespace passed, while one in "prod" failed.
• The demo also showcased dynamic error messages using CEL expressions, allowing the error message to include specific details like the disallowed namespace and the list of allowed namespaces.

1. Helper Function (Image Data Parsing):

• This demo introduced a Kyverno-provided CEL helper function, imageData, which parses an image string into its components (registry, repository, tag).
• The ValidatingPolicy iterated over all containers in a Pod (or Deployment) and used imageData to check if the registry component of each image was gcr.io.
• A deployment using gcr.io images passed, while one using a different registry failed. This demonstrated how specific image properties can be programmatically enforced.

ImageValidatingPolicy Demonstrations

1. Verify Image Signature:

• An ImageValidatingPolicy was presented, matching all Pods and specifically targeting images in the gcr.io registry.
• An attester of type notary was defined, providing a certificate value for verification.
• The validation rule used the verifyImageSignature CEL function, passing the image and the defined notary attester. The policy asserted that the return value (count of successful verifications) was greater than zero.
• A signed image passed, while an unsigned image failed, demonstrating the enforcement of image authenticity.

1. Attestation Verification (SBOMs):

• A similar ImageValidatingPolicy was shown, but this time verifying an SBOM attestation (using the OCI referrers API).
• The validation rule employed the verifyAttestationSignature CEL function, passing the image, the attestation type, and the attester.
• The policy successfully validated an image with a signed SBOM attestation.

1. Attestation Payload Verification (SBOM Format):

• Building on the previous demo, this policy added a second condition to verify the payload of the SBOM attestation.
• It fetched the attestation payload and checked if the sbom.format field was set to cycloneDX.
• The policy passed when the SBOM format matched cycloneDX but failed when the format was intentionally changed to "temp something" in the example, showcasing granular control over attestation content.

The playground environment allowed for real-time interaction, policy modification, and immediate feedback, providing a clear and tangible proof of concept for the new Kyverno features.

Defensive Implications

▶ Watch: Kyverno's Cleanup policies for removing unused resources (8:00)

The advancements in Kyverno, particularly the adoption of CEL and the introduction of new CRDs, have profound implications for Kubernetes security and defense.

Firstly, the standardization on CEL significantly lowers the barrier to entry for security practitioners. As Kubernetes itself now uses CEL for its native Validating Admission Policies, a unified language reduces cognitive load and promotes consistency across policy definitions. This means defenders can leverage a widely understood and powerful expression language, reducing the learning curve and potential for misconfigurations when writing complex policies.

Secondly, the extended CEL capabilities enable a new generation of dynamic and context-aware policies. The ability to perform resource lookups allows policies to make decisions based on the state of other cluster resources. For example, a policy could ensure that deployments only use secrets from a specifically approved ConfigMap, or that network policies are automatically generated for new namespaces based on a template defined elsewhere. The external service calls via http.get are a game-changer, allowing policies to integrate with external security tools, threat intelligence feeds, or compliance databases in real-time. This means a policy could dynamically block an image if its hash appears on a blacklist maintained by an external service, or if a vulnerability scanner reports critical CVEs not yet present in an internal database.

Thirdly, the dedicated ImageValidatingPolicy CRD and its advanced features are critical for bolstering supply chain security. By enforcing image signatures from trusted authorities like Notary and Cosign, organizations can prevent the deployment of untrusted or tampered images. More importantly, the ability to verify attestation signatures and inspect attestation payloads (e.g., SBOMs, vulnerability reports) allows for highly granular control. Defenders can mandate that all deployed images must have a signed SBOM in a specific format (e.g., CycloneDX) and crucially, that these SBOMs must not list any critical or high-severity vulnerabilities. This moves beyond simple image signing to a proactive, evidence-based approach to software integrity, enabling automated checks against the actual components and security posture of container images.

Finally, the support for any JSON payload extends Kyverno's defensive perimeter beyond the Kubernetes cluster. Security teams can now apply the same robust policy engine to validate configuration files, CI/CD pipeline definitions, or other JSON artifacts, ensuring compliance and security standards are met throughout the software development lifecycle, not just at deployment time. This provides a holistic approach to governance, unifying policy enforcement across diverse cloud-native components. The new CRDs also enhance clarity, making it easier to audit and manage specific types of policies, improving the overall maintainability of the security posture.

Key Takeaways

• Kyverno's API is being simplified and standardized: New, specialized CRDs like ValidatingPolicy and ImageValidatingPolicy are replacing the monolithic ClusterPolicy, making policy definitions clearer and more focused.
• CEL is the new standard for policy expressions: Kyverno is fully embracing Kubernetes' Common Expression Language (CEL), providing a powerful, side-effect-free, and widely adopted language for policy authoring.
• CEL is extended with powerful Kyverno-specific capabilities: Custom CEL libraries enable advanced features like resource lookups, external service calls (http.get), and ConfigMap lookups, ensuring Kyverno retains and expands its unique functionalities beyond native Kubernetes VAPs.
• Enhanced image verification for robust supply chain security: The ImageValidatingPolicy offers dedicated support for verifying image and attestation signatures (Notary, Cosign) and, critically, allows for deep inspection of attestation payloads (e.g., SBOMs, vulnerability reports) to enforce granular security requirements.
• Policy enforcement extends beyond Kubernetes: With evaluationMode: JSON, Kyverno can now apply policies to any JSON-based payload, enabling governance and security checks for non-Kubernetes artifacts and configurations across the cloud-native ecosystem.
• New features are available for immediate testing: A Kyverno playground environment and example policies are available for users to experiment with the new ValidatingPolicy and ImageValidatingPolicy CRDs in Kyverno 1.14.

About the Speaker(s)

Frank Jogeleit is a Senior Software Engineer at Nuvata. His work focuses on cloud-native technologies, particularly within the Kubernetes ecosystem. In this talk, Frank provided insights into the architectural motivations behind Kyverno's evolution and showcased its new ValidatingPolicy features through live demonstrations.

Vishal Choudhary is a Kyverno Maintainer, deeply involved in the development and community aspects of the project. Vishal elaborated on Kyverno's current capabilities, the technical details of the new ImageValidatingPolicy, and demonstrated its advanced image verification features, highlighting its role in securing the software supply chain.