What's New in gRPC - Kevin Nilson, Google

What's New in gRPC

Speakers: Kevin Nilson, Google; Gina, Google; Richard, Google

Conference: KubeCon EU

YouTube: https://www.youtube.com/watch?v=0qNOZpdW870

Overview

This talk, presented by gRPC maintainers Kevin Nilson, Gina, and Richard from Google at KubeCon EU, provides a comprehensive update on the latest developments, features, and community initiatives within the gRPC ecosystem. It highlights gRPC's substantial growth and widespread adoption, evidenced by millions of weekly downloads across major languages like Java, Python, and Node. The core of the presentation focuses on significant technical advancements aimed at enhancing gRPC's capabilities in cloud-native environments, particularly around service mesh integration, observability, resilience, and API evolution.

The speakers underscore a strong commitment to addressing community feedback, with notable efforts in improving documentation and introducing long-requested features such as REST support. The talk details a roadmap for upcoming functionalities, emphasizing how gRPC is evolving to meet the complex demands of modern distributed systems, from granular traffic management with Kubernetes Gateway API to advanced rate limiting and seamless integration with serverless platforms. This update is crucial for developers and architects leveraging gRPC in their microservices architectures, offering insights into tools and strategies for building more robust, observable, and scalable applications.

The presentation also touches upon gRPC's journey towards CNCF graduation, signaling a push for more open governance and sustained community collaboration. For anyone invested in the future of inter-service communication in the cloud, this talk serves as an essential guide to the current state and exciting trajectory of gRPC, demonstrating its continued relevance and innovation in the cloud-native landscape.

Background

▶ Watch: Introduction and impressive gRPC growth statistics (0:00)

gRPC, a high-performance, open-source universal RPC framework developed by Google, has become a cornerstone for building distributed systems and microservices, particularly within the cloud-native ecosystem. Its foundation on Protocol Buffers for efficient serialization and HTTP/2 for multiplexing, streaming, and header compression offers significant advantages over traditional REST APIs in terms of performance and developer experience.

Prior to these updates, gRPC had already established itself as a critical component for many organizations, facilitating seamless communication between services. However, as with any rapidly evolving technology, community feedback is vital for its continued improvement. A recurring theme from past KubeCon events and community discussions highlighted two primary areas for enhancement: documentation quality and the desire for native REST support, acknowledging that while gRPC excels in inter-service communication, REST remains prevalent for external APIs and browser-based interactions.

The project has also been an incubating project within the Cloud Native Computing Foundation (CNCF). As an incubating project, gRPC has been working towards graduation, a status that signifies a project's maturity, healthy governance, and broad adoption. This process often involves refining project governance structures to ensure long-term sustainability and community engagement. The efforts detailed in this talk build upon gRPC's existing strengths, addressing these known community needs and pushing the boundaries of what's possible in cloud-native service communication.

Key Findings

▶ Watch: Announcing upcoming gRPC support for REST (2:10)

The talk unveiled a wealth of new features and strategic directions for gRPC, reflecting a strong commitment to community feedback, enhanced developer experience, and robust cloud-native integration. Key findings include:

• Significant Growth and Community Engagement: gRPC continues to experience exponential growth, with millions of weekly downloads across Java, Python, and Node, and a consistently upward trend in GitHub stars.
• Documentation Overhaul: A major focus has been placed on improving documentation, including new sections, numerous short YouTube videos, and updated code examples, directly addressing prior community feedback.
• Planned REST Support: A highly requested feature, native REST support, is on the roadmap with a preview release anticipated at gRPC Conf, aiming to bridge gRPC's performance with REST's ubiquity.
• CNCF Graduation Efforts: gRPC is actively working towards graduating from a CNCF incubating project, involving a significant rewrite of its governance model and steering committee structure to foster a more open and sustainable project.
• Enhanced Service Mesh Capabilities: A suite of features designed to deepen gRPC's integration with service meshes, including:
• Kubernetes Gateway API gRPC Route (GA): Providing granular traffic management for gRPC services within Kubernetes.
• OpenTelemetry Observability: First-class support for OpenTelemetry metrics (latency, QPS, error rates, payload sizes) and tracing across all gRPC languages.
• Stateful Session Affinity (gRPC C++): A load balancing technique using cookies to route client requests to the same backend, crucial for stateful applications.
• Dual Stack Backend Support: Enabling gRPC clients to connect to backends with both IPv4 and IPv6 addresses, enhancing network flexibility.
• XDS Fallback: A mechanism for automatic failover to a secondary XDS control plane, ensuring continuous operation and resilience.
• Proxyless Service Mesh on Cloud Run: Features enabling seamless interoperability between Kubernetes and serverless platforms like Google Cloud Run, including XDS host rewriting, Jot token-based authorization, and mTLS O based on Spiffy identities.
• XDS Global Rate Limiting: A groundbreaking, completely open-source solution for global rate limiting, natively implemented in the gRPC library using the existing XDS protocol and a new RLQS protocol for asynchronous policy programming and decentralized data plane.
• Protobuf Editions: Introduction of a versioning system for Protocol Buffer features (e.g., Proto edition 2023), offering snapshots of features with customizable settings, ensuring forward compatibility and simplifying updates. A new Protoiller CLI tool is also in development for migration assistance.

These findings collectively highlight gRPC's evolution into an even more powerful, flexible, and resilient framework for cloud-native application development, with a clear focus on developer needs and operational excellence.

Technical Deep Dive

▶ Watch: gRPC Conf 2024: Date, location, and call for speakers (3:45)

The technical advancements discussed in the gRPC maintainer talk span several critical areas, from core protocol enhancements to deep integration with cloud-native infrastructure.

Documentation and Community Initiatives

Addressing a major pain point, gRPC has significantly ramped up its documentation efforts. Over the last two years, the project has introduced four new documentation sections, published 39 new videos, and added a multitude of new code examples. The strategy has shifted towards creating short, consumable YouTube videos to provide quick overviews and getting-started guides for new features. For critical announcements, such as outages or security patches, a dedicated mailing list, gRPCIO-announce, has been established, separate from the more general gRPCIO list for social interaction and questions.

REST Support Roadmap

Recognizing the community's strong desire for better interoperability with RESTful services, gRPC is actively developing native REST support. The roadmap includes plans for a pre-alpha preview release with hands-on code labs at gRPC Conf later in the year, with a full feature rollout planned subsequently. This initiative aims to bridge the gap between gRPC's performance benefits and the widespread adoption of REST for certain use cases.

CNCF Graduation and Governance

gRPC is currently an incubating project within the CNCF and is working towards graduation. This process involves a significant rewrite of the project's governance model, including key changes to the steering committee and other governance items. These changes are being driven in collaboration with the CNCF to enhance the project's openness, sustainability, and community engagement.

Service Mesh Enhancements

A substantial portion of the talk focused on bolstering gRPC's capabilities within service mesh environments, emphasizing ease of adoption and advanced traffic management.

• Kubernetes Gateway API gRPC Route: The Kubernetes Gateway API has announced the gRPC Route resource in General Availability (GA), supported across Google Cloud Platform and other cloud providers. This powerful feature allows users to define sophisticated routing rules for gRPC applications, enabling precise control over how requests are directed across backend services. Criteria such as service methods and headers can be leveraged to match incoming requests, facilitating advanced traffic management strategies like canary deployments, A/B testing, and traffic splitting for gRPC services.

• OpenTelemetry Observability: gRPC is expanding its observability support to OpenTelemetry telemetry. This includes metrics available in all supported gRPC languages, providing insights into RPC latency, QPS (Queries Per Second), error rates, and payload sizes. The talk confirmed that this is a first-class, native OpenTelemetry instrumentation, moving beyond previous third-party interceptors or stats handlers. OpenTelemetry tracing is also now available, encouraging developers to integrate it for better troubleshooting and system management.

• Stateful Session Affinity (gRPC C++): Introduced for gRPC C++, Stateful Session Affinity is a load balancing technique designed to ensure that all requests from a particular client are consistently routed to the same backend server. This is particularly useful for applications maintaining per-session state, such as shopping carts or user profiles. The implementation leverages HTTP cookies:

1. The first request from a client is routed based on standard load balancing policies.
2. The chosen server encodes its identity into a cookie and includes it in the Set-Cookie response header.
3. The gRPC client receives this cookie and attaches it to all subsequent requests within that session.
4. gRPC ensures these subsequent requests are routed to the same server as long as the cookie is valid or the server is available.

This guarantees a "warm cache" for the session, significantly speeding up applications. Configuration is managed via a custom resource called GCP session affinity policy, where users can set cookie TTL (time-to-live) in seconds and specify the targetReference for which route or service to apply the policy.

• Dual Stack Backend Support: While gRPC clients have long supported both IPv4 and IPv6, many language implementations previously lacked support for individual backends having both IPv4 and IPv6 addresses. This new feature addresses that limitation, allowing the resolver and LB policy API to support multiple addresses per endpoint. It employs Happy Eyeballs to minimize the time taken to determine the optimal address, enhancing network flexibility and resilience. This support is now available across all gRPC languages.

• XDS Fallback: To further enhance reliability and resilience, gRPC has introduced XDS Fallback. This capability allows users to configure and deploy a secondary control plane alongside the primary one. In scenarios where the primary control plane becomes unavailable (due to maintenance or network issues), gRPC automatically switches over to the secondary control plane. This automatic failover mechanism ensures continuous service operation, minimizing downtime and maintaining a consistent user experience.

• Proxyless Service Mesh on Cloud Run: This bundle of features aims to improve interoperability between Kubernetes-based service meshes and serverless platforms like Google Cloud Run. Historically, bridging these environments for service mesh has been challenging. The new features make this a first-class experience:
• XDS host rewriting: Enables the use of vanity URLs in XDS targets, common in serverless platforms.
• Jot token-based authorization: Provides secure authorization for outgoing service mesh-mediated RPCs.
• mTLS O based on Spiffy identities: Enhances mutual TLS authentication using Spiffy identities for secure communication.

These features, initially coming to Java and later other languages, enable seamless and secure communication between services running on Kubernetes and Cloud Run.

XDS Global Rate Limiting

A particularly exciting new capability, publicly announced for the first time, is XDS Global Rate Limiting for gRPC Proxyless. This addresses situations where servers are frequently overwhelmed by requests, offering a more robust alternative to proxy-based rate limiting. Key aspects include:

• Native Implementation: gRPC servers natively implement rate limiting functionality directly within the gRPC library, in conjunction with a control plane.
• Decentralized Data Plane, Global Decisions: It leverages the existing XDS protocol for service mesh configuration and introduces a new RLQS (Rate Limit Query Service) protocol. RLQS is specifically designed for aggregating load information from gRPC servers and asynchronously programming rate limiting policy from a globalized control plane. This allows for globally aware rate limiting decisions while maintaining a completely decentralized data plane.
• How it Works:

1. The XDS control plane sends configuration to the gRPC server, instructing it to connect to an RLQS server.
2. The gRPC server connects to the RLQS control plane.
3. The server groups requests into buckets based on their metadata.
4. Periodically, the gRPC server sends usage reports for each bucket to the RLQS control plane.
5. Asynchronously, the RLQS control plane sends query per second (QPS) limits down to the gRPC server for each bucket.
6. If the QPS rate for any bucket exceeds its maximum, the gRPC server rejects the request with a configurable status code.

This system offers a highly performant and globally aware solution for managing high-volume systems. The protocols and gRPC library implementation are open source, with a first RLQS control plane implementation slated for GCP Cloud Service Mesh in the coming months.

Protobuf Editions

The talk introduced Protobuf Editions as a new versioning system for Protocol Buffer features. Unlike previous separate syntaxes like Proto2 and Proto3 with fixed rule sets, Editions provide snapshots of Protobuf features with customizable settings. This approach ensures forward compatibility, meaning code written in older editions will continue to work with newer ones. By unifying features and enabling incremental updates, Editions simplify maintenance and provide greater flexibility. Proto edition 2023 is the first edition, essentially combining features from Proto2 and Proto3, and is supported in many languages. Adoption requires a few code changes, detailed in public documentation. A new CLI tool, Protoiller, is also in development to assist with migrating files, messages, and fields to new values for each feature. A preview of Protobuf 2024 indicates future cross-language features like symbol visibility and style guide enforcement, alongside language-specific improvements.

Demo / Proof of Concept

▶ Watch: gRPC service mesh streamlines deployment and features (7:00)

While the talk announced plans for "hands-on code labs" and a "preview release" for REST support at the upcoming gRPC Conf, no live demonstration or specific proof of concept was presented during this KubeCon EU session. The focus was primarily on announcing new features and roadmaps.

Defensive Implications

▶ Watch: Kubernetes Gateway API gRPC Route resource announced in GA (8:20)

The new features and updates in gRPC offer significant defensive implications for building more secure, resilient, and observable cloud-native applications:

• Enhanced Reliability with XDS Fallback: The ability to configure a secondary XDS control plane provides a critical layer of defense against control plane outages, ensuring continuous service operation and minimizing downtime during unexpected failures or planned maintenance. This directly translates to higher availability and fault tolerance for gRPC-based microservices.
• DDoS and Overload Protection with XDS Global Rate Limiting: The introduction of native, global rate limiting within the gRPC library is a powerful defense mechanism against denial-of-service (DoS) attacks and service overload. By allowing granular QPS limits per metadata-defined bucket, services can gracefully reject excessive requests, preventing cascading failures and maintaining stability under high load. This is a crucial tool for maintaining service health and preventing resource exhaustion.
• Granular Traffic Control for Secure Deployments via gRPC Route: The Kubernetes Gateway API's gRPC Route resource empowers defenders with precise control over traffic flow. This enables secure deployment strategies like canary deployments and A/B testing, allowing new versions or features to be rolled out to a small subset of users first. This minimizes the blast radius of potential vulnerabilities or bugs, enhancing the overall security posture and stability of deployments.
• Improved Observability for Rapid Incident Response: First-class OpenTelemetry metrics (RPC latency, QPS, error rate, payload sizes) and tracing provide deep insights into application behavior. This enhanced observability is invaluable for quickly detecting anomalies, identifying the root cause of issues, and responding to security incidents. Faster troubleshooting reduces mean time to recovery (MTTR) and strengthens the ability to proactively monitor for suspicious activity.
• Secure Hybrid Deployments with Proxyless Service Mesh on Cloud Run: The features enabling seamless gRPC communication between Kubernetes and serverless platforms like Cloud Run, including Jot token-based authorization and mTLS O based on Spiffy identities, significantly enhance security in hybrid cloud environments. These mechanisms ensure authenticated and encrypted communication, extending the robust security posture of a service mesh to diverse deployment models.
• Increased Resilience with Dual Stack Backend Support: By supporting both IPv4 and IPv6 addresses for backends, gRPC applications gain greater network flexibility and resilience. This ensures services remain accessible even as network infrastructures evolve or if specific IP versions experience issues, reducing potential points of failure.
• Proactive Security Information via gRPCIO-announce: Subscribing to the gRPCIO-announce mailing list is a simple yet effective defensive measure. This channel provides critical notifications about security patches, vulnerabilities, and outages directly from the gRPC maintainers, enabling timely updates and preventative actions to protect applications.
• Session State Management for Robust Applications with Stateful Session Affinity: While primarily a performance feature, ensuring clients consistently hit the same backend for stateful sessions indirectly contributes to application robustness. By maintaining consistent state and leveraging warm caches, the likelihood of unexpected application behavior or errors due to inconsistent state is reduced, improving overall reliability.
• Forward Compatibility and Maintainability with Protobuf Editions: Protobuf Editions, by ensuring forward compatibility and providing a structured way to manage feature sets, simplifies the process of keeping gRPC services up-to-date. This reduces the friction associated with upgrades, making it easier for developers to adopt newer versions that often include security enhancements and bug fixes.

In summary, the latest gRPC updates provide a comprehensive toolkit for defenders, enabling them to build, deploy, and operate services that are not only high-performing but also inherently more secure, resilient, and observable in complex cloud-native environments.

Key Takeaways

• Massive Growth & Community Focus: gRPC continues its exponential growth across major languages, fueled by a strong commitment to addressing community feedback, particularly through improved documentation and a roadmap for native REST support.
• Advanced Service Mesh Integration: gRPC deeply integrates with cloud-native service meshes, offering granular traffic control via Kubernetes Gateway API gRPC Route, first-class OpenTelemetry observability, Stateful Session Affinity for C++, and robust XDS Fallback for high availability.
• Seamless Hybrid Cloud Connectivity: New features enable first-class proxyless service mesh interoperability between Kubernetes and serverless platforms like Google Cloud Run, incorporating XDS host rewriting, Jot token-based authorization, and mTLS with Spiffy identities.
• Groundbreaking Global Rate Limiting: The introduction of XDS Global Rate Limiting, leveraging the new RLQS protocol, provides native, decentralized, and globally aware overload protection directly within the gRPC library, a critical defense mechanism for high-volume systems.
• Evolving Protocol Buffers: Protobuf Editions introduce a versioning system for features, ensuring forward compatibility and simplifying updates, with Proto edition 2023 unifying previous syntaxes and a Protoiller tool in development for migration.
• Commitment to Open Governance: gRPC is actively working towards CNCF graduation, involving a significant rewrite of its governance model to foster a more open and sustainable project for the long term.

About the Speaker(s)

The talk was delivered by three gRPC maintainers from Google:

• Kevin Nilson: Kevin kicked off the session, providing an overview of gRPC's impressive growth and the project's dedication to improving documentation based on community feedback. He also highlighted the upcoming gRPC Conf and the gRPCIO-announce mailing list for critical updates.
• Gina: Gina, who was celebrating her 10-year anniversary at Google on the Monday following the talk, focused on gRPC's cloud-native adoption features. She detailed advancements in service mesh capabilities, including the Kubernetes Gateway API gRPC Route, OpenTelemetry support, Stateful Session Affinity in gRPC C++, Dual Stack Backend Support, and XDS Fallback.
• Richard: Richard presented the latest innovations around proxyless service mesh on serverless platforms like Google Cloud Run, outlining features such as XDS host rewriting, Jot token-based authorization, and mTLS O with Spiffy identities. He also introduced the groundbreaking XDS Global Rate Limiting and provided insights into the future of Protocol Buffers with Protobuf Editions.

All three speakers are integral to the gRPC project at Google, contributing to its development, maintenance, and strategic direction.