WG-Batch Updates: What’s New and What Is Next? - Marcin Wielgus, Google
Marcin Wielgus, Google
KubeCon + CloudNativeCon Europe 2025 · Session
Marcin Wielgus, an organizer of the Kubernetes Batch Working Group, presented a comprehensive update on the group's efforts to enhance Kubernetes' support for batch workloads. This presentation, titled "WG-Batch Updates: What’s New and What Is Next?", highlighted the significant advancements in Kueue, a powerful workload-level scheduler, alongside updates to core Kubernetes APIs like Job and new APIs such as JobSet and K-job. The talk underscored the Batch Working Group's mission to reduce fragmentation within the Kubernetes ecosystem, making it a more robust and efficient platform for high-performance computing (HPC), artificial intelligence (AI), machine learning (ML), data analytics, and continuous integration (CI) tasks.
AI review
This KubeCon update from the Kubernetes Batch Working Group, led by Marcin Wielgus, delivers a highly substantive deep dive into Kueue and related APIs, directly addressing the long-standing challenges of running HPC, AI/ML, and data analytics workloads on Kubernetes. The talk details advanced scheduling mechanisms like topology-aware placement, sophisticated fair-sharing models (both preemption and admission-based), and hierarchical quotas, alongside significant enhancements to the core Job API, JobSet, and the new Slurm-like K-job tool. It provides actionable insights and critical signal…