CNCF TAG Network and Cloud Native Network Landscape - Zhonghu Xu, Huawei & Nic Jackson, Hashicorp

CNCF TAG Network and Cloud Native Network Landscape - Zhonghu Xu, Huawei & Nic Jackson, Hashicorp

Speakers: Zhonghu Xu, Open Source Software Engineer, Huawei; Nic Jackson, Hashicorp

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=gMDC1zzHabk

Overview

This talk, presented by Zhonghu Xu of Huawei and Nic Jackson of Hashicorp, delves into the current state and future evolution of the CNCF TAG Network (Technical Advisory Group Network). It provides a comprehensive overview of the diverse projects under the TAG's purview, ranging from mature graduated projects like Cilium and Linkerd to innovative sandbox projects challenging established paradigms. A significant portion of the discussion focuses on recent advancements in cloud-native networking, particularly around service mesh technologies and multi-cluster management, highlighting projects like KMesh and Semant that offer low-overhead alternatives to traditional sidecar architectures.

The presentation also addresses a pivotal change within the CNCF: the impending TAG reboot. This restructuring will see TAG Network merge into a broader TAG Infrastructure, aiming to streamline operations, foster community engagement, and encourage more focused initiatives and whitepaper generation. The speakers articulate the rationale behind this reboot, emphasizing the need to alleviate administrative burdens on TAG leadership and re-focus on community knowledge sharing. This talk is crucial for anyone involved in cloud-native infrastructure, network architects, and developers looking to understand the cutting edge of networking within Kubernetes and how to contribute to its future direction.

The importance of this discussion extends beyond mere project updates; it offers a critical lens into the evolving challenges and solutions in the cloud-native networking space. As applications become increasingly distributed and complex, the need for efficient, secure, and scalable networking solutions is paramount. The projects highlighted, especially those leveraging technologies like eBPF and bytecode manipulation, signify a continuous drive towards optimizing performance and resource utilization without compromising on the robust features expected from modern networking stacks. Furthermore, the emphasis on community involvement and the restructuring of TAGs underscore the collaborative nature of the CNCF ecosystem in addressing these complex problems.

Background

▶ Watch: Welcome and TAG Network reboot announcement (0:00)

The Cloud Native Computing Foundation (CNCF) organizes its technical guidance and project oversight through Technical Advisory Groups (TAGs). These long-lived groups report to the Technical Oversight Committee (TOC) and serve several critical functions. Their responsibilities include bridging the gap between end-users, project contributors, and TOC members; identifying missing functionalities or gaps within the CNCF project landscape; educating and informing users with unbiased, practical information; fostering project maturity from sandbox to incubation and graduation; clarifying relationships between overlapping projects (e.g., multiple API gateways); and engaging wider communities to facilitate contributions. Essentially, TAGs act as a vital conduit for technical direction, community building, and project governance within the CNCF.

The problem addressed by the upcoming TAG reboot stems from several challenges identified within the existing structure. As Nic Jackson explained, being a co-chair or tech lead for a TAG often entails significant time commitment, typically in addition to full-time day jobs. This has led to burnout, a loss of experienced domain experts, and leadership attrition over time. Furthermore, some TAGs found themselves predominantly focused on administrative tasks, such as processing sandbox applications, rather than fulfilling their core mission of fostering knowledge, producing white papers, and guiding technical direction for the community. The work often extended beyond the originally intended scope, leading to inefficiencies and reduced utility for the broader community.

To address these issues, the CNCF is undergoing a significant TAG reboot. The current eight TAGs are being consolidated into five streamlined groups: Developer Experience, Workloads, Foundation Infrastructure (which will absorb TAG Network), Operational Resilience, and Security & Compliance. This restructuring aims to create a more sustainable and impactful framework. The new model will place greater emphasis on initiatives—short-term, research-oriented projects often culminating in white papers or defined patterns—and sub-projects for longer-term, more foundational work (e.g., AI initiatives). This shift intends to re-energize community involvement, make the TAGs more responsive to emerging technical needs, and better support the TOC by sharing the workload related to project evolution and technical guidance.

Key Findings

▶ Watch: Detailed explanation of TAG Network's responsibilities (2:10)

The presentation highlighted several key findings and trends within the cloud-native networking landscape, particularly as reflected by projects under TAG Network (soon to be TAG Infrastructure):

1. Diverse and Maturing Project Ecosystem: The CNCF boasts a robust and diverse set of networking projects. Graduated projects like Cilium (eBPF-based networking and security), Linkerd (ultralight service mesh), and CoreDNS (flexible DNS server) demonstrate established, production-ready solutions. Incubation projects such as Kuma (universal service mesh), Emissary Ingress (API gateway, formerly Ambassador), Envoy Gateway (declarative API gateway built on Envoy), and gRPC (high-performance RPC framework) show a strong pipeline of maturing technologies.

1. Innovation in the Sandbox: The sandbox stage is a hotbed of innovation, with many projects challenging traditional patterns and addressing complex, modern cloud-native challenges. Key themes include:

• Multi-cluster Management and Load Balancing: Projects like Lockb focus on multi-cluster load balancing and redundancy, addressing the increasing complexity of geographically distributed applications. K8GB (Kubernetes Global Server Load Balancing) is also staging for incubation, indicating a strong community interest in global traffic management.
• Enhanced Networking Capabilities: Quadrant aims to provide advanced networking features, such as allocating IP addresses within a namespace inside a pod, offering finer-grained control.
• Secure Multi-cluster Traffic Flow: CubeSlice enables secure traffic flow across multiple Kubernetes clusters, allowing policy definition and supporting multi-cluster failover for enhanced resilience in regional setups.
• Challenging Service Mesh Paradigms: This is perhaps the most significant trend. Projects like Semant and KMesh are actively seeking alternatives to the traditional sidecar pattern for service meshes, aiming for lower latency and reduced overhead.
• Semant achieves this for Java applications by modifying bytecode, integrating service mesh capabilities directly into the application runtime without a separate sidecar proxy. It's designed to work alongside existing Istio deployments, offering selective low-overhead management.
• KMesh leverages eBPF to implement its service mesh data plane, promising high performance and minimal resource consumption. Its demo showcased dynamic header routing and weighted traffic splitting, fundamental service mesh features, implemented with eBPF.
• API Evolution and Compatibility: Connect offers a compelling solution for API developers, providing gRPC compatibility while also exposing a plain HTTP/JSON API without requiring additional code. This simplifies client-side interactions while retaining the performance benefits of gRPC.
• New Ingress/Gateway Solutions: K Gateway (formerly Glue from Solo.io) is the latest submission, indicating continued evolution in the API gateway space.

1. The CNCF TAG Reboot: The most overarching finding is the impending restructuring of CNCF TAGs. This move, driven by the need to combat burnout, loss of expertise, and administrative overload, will consolidate eight TAGs into five. The goal is to shift focus from administrative tasks to community-driven initiatives and the creation of valuable resources like white papers and best practices. TAG Network will merge into TAG Infrastructure, signaling a more integrated approach to foundational cloud-native components.

These findings collectively point to a dynamic and rapidly evolving cloud-native networking landscape, characterized by a pursuit of efficiency, scalability, and enhanced control, all while adapting to the growing complexity of distributed systems.

Technical Deep Dive

▶ Watch: TAG Network's mission: educating users and fostering community (4:10)

The technical core of the presentation centered on the evolution of service mesh architectures and innovative networking solutions, particularly highlighting how new projects are challenging established patterns.

Traditionally, service meshes like Istio and Linkerd predominantly rely on the sidecar pattern. In this model, a small proxy (e.g., Envoy for Istio) runs as a separate container alongside each application pod. This sidecar intercepts all inbound and outbound network traffic for the application, enforcing policies, collecting telemetry, and handling features like mTLS, retries, and circuit breaking. While powerful, sidecars introduce overhead in terms of resource consumption (CPU, memory) and latency due to the extra hop and context switching.

The talk introduced two significant projects aiming to mitigate these sidecar drawbacks: Semant and KMesh.

Semant takes an innovative approach specifically for Java applications. Instead of a sidecar, Semant integrates service mesh capabilities directly into the application's runtime by leveraging Java bytecode manipulation. This means that networking concerns like traffic routing, policy enforcement, and observability are handled within the application process itself, without requiring an external proxy. The key advantages are extreme low latency and low overhead, as there's no inter-process communication or additional container to manage. Crucially, Semant is designed to augment, rather than replace, existing service mesh deployments. For instance, it can work alongside an Istio deployment, allowing users to select specific Java applications that require the ultra-low latency benefits of Semant, while other workloads continue to use Istio's sidecar or ambient mesh (Istio's newer, sidecar-less data plane option) patterns. This offers a flexible path for optimization.

KMesh represents another frontier in low-overhead service mesh data planes, utilizing eBPF (extended Berkeley Packet Filter). eBPF allows programs to run in the Linux kernel without modifying kernel source code or loading kernel modules. This provides highly efficient, programmable packet processing capabilities. KMesh leverages eBPF to implement its data plane, moving traffic management logic directly into the kernel space. This bypasses the need for user-space proxies entirely for data plane operations, significantly reducing context switches and improving performance. The demo explicitly showed KMesh performing dynamic header routing and weighted traffic splitting, demonstrating that complex service mesh features can be efficiently implemented using eBPF, potentially offering superior performance and resource efficiency compared to traditional proxy-based solutions.

Beyond service meshes, the talk highlighted other technically interesting projects:

• Connect: This project addresses the perennial challenge of API interoperability. While gRPC offers high-performance, strongly typed communication, its binary nature can be cumbersome for certain clients or debugging. Connect provides a solution that is fully gRPC compatible but also allows exposing a plain HTTP/JSON API for the same services without requiring any additional code changes. This means developers can write their services once using gRPC definitions and automatically gain both high-performance gRPC endpoints and easily consumable REST-like HTTP/JSON endpoints, bridging the gap between different API consumption patterns.

• CubeSlice: For organizations operating across multiple Kubernetes clusters, multi-cluster networking is a complex problem. CubeSlice tackles this by enabling secure traffic flow and policy enforcement across disparate clusters. It allows defining fine-grained network policies that govern how applications in one cluster can communicate with those in another. Furthermore, it supports multi-cluster failover, an essential feature for building resilient applications that can seamlessly shift traffic to a healthy cluster in case of an outage in another region or cluster. This project is critical for true geo-distributed, highly available cloud-native architectures.

• Lockb: Complementing multi-cluster traffic management, Lockb focuses on multi-cluster load balancing and redundancy. It provides mechanisms to distribute incoming traffic across services deployed in different clusters, ensuring optimal resource utilization and high availability. This is crucial for maintaining performance and resilience in large-scale, distributed environments.

These projects collectively demonstrate a strong technical direction towards more efficient, flexible, and resilient cloud-native networking, driven by innovations in runtime optimization (Semant), kernel-level programmability (KMesh), and sophisticated multi-cluster management (CubeSlice, Lockb).

Demo / Proof of Concept

▶ Watch: Overview of current Graduated and Incubating CNCF projects (6:00)

The core of the technical demonstration focused on KMesh, showcasing its capabilities as an eBPF-based service mesh data plane. The demo illustrated two fundamental service mesh features: dynamic header routing and traffic splitting by weight. Both demonstrations utilized a standard Bookinfo application, a common pattern for showcasing service mesh functionalities.

Demo 1: Dynamic Header Routing

1. Setup: The demo began by ensuring KMesh was installed and the Bookinfo application was deployed. It was emphasized that KMesh operates as an eBPF-based data plane, contrasting it with traditional proxy-based meshes.
2. Waypoint Deployment: A KMesh waypoint was deployed for the reviews service within the Bookinfo application. This waypoint acts as an entry point for KMesh to manage traffic to the reviews service. The waypoint image was explicitly replaced with a kmesh waypoint image.
3. Namespace Labeling: The default namespace, where the Bookinfo application resided, was labeled to be managed by KMesh. This step signals to KMesh which traffic it should intercept and control.
4. Virtual Service Configuration: A virtual service was then created to define the routing logic. The rule stated that if a user logs in as "Jason" (identified via an HTTP header), the traffic should be redirected to reviews V1. For any other user, the traffic would be routed to a different version (implicitly, the default or another configured version).
5. Verification: The presenter explained that this demonstrates a basic yet powerful service mesh capability, implemented using the underlying eBPF technology, allowing granular traffic control based on application-layer attributes like HTTP headers.

Demo 2: Traffic Split by Weight

1. Setup: Similar to the first demo, the Bookinfo application was installed and confirmed to be running healthily.
2. Waypoint Deployment: A KMesh waypoint was again deployed for the reviews service.
3. Weighted Routing Configuration: The routing was configured to split traffic to the reviews service based on weight: 90% of the traffic was directed to reviews V1, and the remaining 10% was sent to reviews V2.
4. Verification: The demonstration used curl commands to repeatedly access the Bookinfo application. The output clearly showed that approximately 90% of the requests were served by reviews V1, and about 10% by reviews V2, visually confirming the successful implementation of weighted traffic splitting.

These demonstrations effectively showcased KMesh's ability to provide core service mesh functionalities, such as intelligent routing and traffic management, with the claimed benefits of an eBPF-based data plane. The visual confirmation of traffic distribution reinforced the project's technical viability and practical utility.

Defensive Implications

▶ Watch: New Sandbox projects addressing multi-cluster networking complexity (7:45)

The insights from this talk offer several crucial defensive implications for organizations operating cloud-native environments:

1. Embrace Advanced Service Mesh Capabilities: The discussion around projects like KMesh and Semant highlights the ongoing evolution of service meshes towards lower overhead and higher performance. Defenders should not only encourage the adoption of established service meshes like Istio or Linkerd for their inherent security benefits (mTLS, policy enforcement, traffic encryption, fine-grained access control) but also keep an eye on these newer technologies. If performance or resource constraints are blocking wider service mesh adoption, solutions like Semant (for Java applications) or eBPF-based meshes like KMesh could offer a path to implement critical security controls without significant performance penalties. This allows for a more comprehensive and pervasive application of security policies across the microservices landscape.

1. Strengthen Multi-cluster Security and Resilience: As applications span multiple Kubernetes clusters, securing inter-cluster communication becomes paramount. Projects like CubeSlice provide essential capabilities for defining explicit security policies and controlling traffic flow between clusters. Defenders should leverage such tools to:

• Isolate Traffic: Enforce network segmentation and least privilege principles across cluster boundaries.
• Define Secure Communication Paths: Ensure that only authorized services can communicate, regardless of their cluster location.
• Implement Multi-cluster Failover: Utilize features like multi-cluster failover to enhance application resilience and ensure continuity during regional outages or cyber-attacks, providing a robust disaster recovery strategy. Lockb further aids in ensuring reliable load balancing across these resilient setups.

1. Re-evaluate API Gateway Security: With new API gateway projects like K Gateway and the evolution of existing ones like Emissary Ingress, defenders must continuously assess their API security posture. API gateways are critical enforcement points for authentication, authorization, rate limiting, and attack surface reduction. Staying updated with these projects ensures that organizations are using the most secure and efficient solutions for exposing their APIs to internal and external consumers. The versatility of Connect also means that while gRPC offers strong typing benefits, the availability of a JSON/HTTP endpoint may introduce different attack vectors that need to be secured consistently.

1. Leverage eBPF for Deeper Visibility and Control: The prominence of eBPF in projects like KMesh underscores its growing importance in cloud-native networking and security. eBPF provides unparalleled visibility into kernel-level network activities and allows for dynamic policy enforcement without modifying the kernel. Defenders should explore how eBPF can be used not just for service mesh data planes but also for advanced network monitoring, intrusion detection, and real-time threat response within Kubernetes environments. This technology offers a powerful primitive for building more secure and observable infrastructure.

1. Active Community Participation in TAG Infrastructure: The restructuring of the CNCF TAGs, particularly the merger of TAG Network into TAG Infrastructure, presents an opportunity for defenders to influence the future of cloud-native networking security. By participating in discussions, contributing to initiatives, and reviewing white papers, security professionals can ensure that security-by-design principles are embedded into new networking patterns and projects from their inception. This proactive engagement is vital for shaping best practices and standards that benefit the entire community.

In summary, the defensive strategy should involve adopting advanced networking controls, prioritizing multi-cluster security, continuously evaluating API security mechanisms, exploring eBPF for enhanced visibility and control, and actively participating in the CNCF community to drive security-focused innovation.

Key Takeaways

• CNCF TAG Network's Evolution: The CNCF TAG Network is undergoing a significant reboot, merging into the broader TAG Infrastructure. This aims to streamline operations, foster community-driven initiatives, and re-focus on generating valuable technical knowledge and white papers for the cloud-native ecosystem.
• Next-Generation Service Meshes: New sandbox projects like Semant and KMesh are challenging the traditional sidecar pattern for service meshes. Semant uses Java bytecode manipulation for low-latency, low-overhead service mesh capabilities, while KMesh leverages eBPF for an efficient, kernel-space data plane. These offer powerful alternatives, particularly for performance-critical applications, and can augment existing Istio deployments.
• Critical Multi-cluster Management: Projects like CubeSlice and Lockb are addressing the increasing complexity of multi-cluster deployments. CubeSlice enables secure traffic flow, policy definition, and multi-cluster failover, crucial for highly available and resilient distributed applications, while Lockb focuses on multi-cluster load balancing and redundancy.
• API Flexibility with gRPC Compatibility: Connect offers an innovative solution for API development, providing full gRPC compatibility while simultaneously exposing plain HTTP/JSON APIs without requiring additional coding effort. This simplifies client integration and debugging while retaining gRPC's performance benefits.
• Community Contribution is Key: The speakers strongly emphasized the importance of community involvement. Whether through contributing specialist knowledge, participating in bi-weekly meetings, joining initiatives, or mentoring underrepresented groups, active participation is vital for shaping the direction and addressing the challenges of cloud-native networking and infrastructure.

About the Speaker(s)

Zhonghu Xu is an Open Source Software Engineer at Huawei, based in China. His work focuses on contributing to and advancing open-source projects, particularly within the cloud-native networking domain, as evidenced by his involvement in the CNCF TAG Network.

Nic Jackson is associated with Hashicorp and serves as a co-chair for the CNCF TAG Network. In this leadership role, he helps guide the technical direction of networking projects within the CNCF, fosters community engagement, and plays a key part in the ongoing restructuring and evolution of the TAGs.