Venn: Resource Management for Collaborative Learning Jobs

Venn: Resource Management for Collaborative Learning Jobs

Speakers: Jiachen Liu, Fan Lai, Eric Ding, Yiwen Zhang, Mosharaf Chowdhury

Conference: MLSys 2025

YouTube: https://www.youtube.com/watch?v=None

Overview

This talk introduces Venn, a novel resource manager designed to optimize the execution of multiple concurrent collaborative learning jobs across large-scale, heterogeneous edge devices. Presented by Jiachen Liu from the University of Michigan, Venn addresses the increasingly critical challenge of managing shared resources in environments where numerous machine learning tasks, particularly those leveraging private user data, compete for limited and unpredictable device availability. The core problem Venn tackles is the complex resource contention that arises when companies deploy many collaborative learning jobs, such as federated learning, on diverse user devices like smartphones.

The motivation for Venn stems from the limitations of large AI models, which, despite their power, often fall short in achieving true personalization and adapting to specialized tasks, especially when private user data is involved. Collaborative learning paradigms offer a solution by enabling models to learn from distributed, private data directly on user devices without centralizing sensitive information. However, current resource management solutions for these scenarios are either designed for single-job environments or employ oversimplified strategies like random device matching, failing to account for the intricate contention patterns that significantly impact job completion times. Venn proposes a contention-aware scheduling and resource-aware matching framework to overcome these limitations, promising substantial speedups in the overall training process for collaborative learning jobs.

The significance of Venn lies in its potential to unlock the full value of private, distributed user data for building highly personalized and specialized AI models. By efficiently managing the unpredictable and heterogeneous nature of edge device resources, Venn allows organizations to run more collaborative learning experiments concurrently, accelerate model development, and ultimately deliver superior user experiences. The approach moves beyond simplistic resource allocation, introducing principles that prioritize jobs based on resource scarcity and demand, a critical advancement for scaling collaborative AI initiatives in real-world deployments.

Background

▶ Watch: Introduction to Venn and collaborative learning challenges (0:00)

The proliferation of large AI models has demonstrated immense capabilities, yet their reliance on public datasets often limits their efficacy in specialized tasks requiring true personalization. This is particularly evident when private user data, which holds significant value for tailoring models to individual needs, cannot be moved off-device due to privacy concerns, regulatory compliance, or data sovereignty. Collaborative learning, exemplified by Federated Learning (FL), emerges as a powerful paradigm to address this, enabling models to be trained on distributed user data directly on edge devices (e.g., smartphones, tablets) without centralizing the raw data. This approach is already prevalent in daily applications from tech giants like Google and Apple for features such as speech recognition and next-word prediction.

However, deploying and managing multiple concurrent collaborative learning jobs at scale introduces substantial challenges that existing resource management solutions largely fail to address. The fundamental issues stem from two core characteristics of the collaborative learning environment:

1. Uncertain and Heterogeneous Edge Devices: Unlike stable cloud resources, edge devices are inherently unreliable. They are typically only available for training when specific conditions are met, such as being connected to Wi-Fi or actively charging. Beyond availability, these devices exhibit significant heterogeneity in their hardware capabilities (e.g., CPU, memory), software versions, and the quantity and nature of data they possess. This variability makes uniform resource allocation nearly impossible and complex to optimize.

1. Diverse Collaborative Learning Job Requirements: Different collaborative learning tasks have distinct resource requirements. A job training a next-word prediction model might require access to keyboard data, which is widely available across devices. In contrast, a job for emoji prediction might only be eligible for devices with specific emoji usage data, representing a much scarcer resource pool. These varying demands, coupled with minimum hardware specifications, create intricate contention patterns when multiple jobs compete for the same pool of devices.

Existing resource scheduling solutions fall short in this complex landscape. Many are designed for single-job scenarios, assuming an abundance of resources and focusing solely on optimizing the "response collection time" – the duration it takes for a job to gather enough devices once it has started. These solutions completely overlook the scheduling delay, which is the time a job spends waiting to acquire enough eligible devices before it can even begin collecting responses. As the demand for collaborative learning grows, this scheduling delay becomes a dominant and unavoidable component of the overall job completion time.

Furthermore, even large companies like Meta, Apple, and Google, despite operating multi-job collaborative learning platforms, often employ oversimplified resource management strategies. These typically involve random device matching algorithms, which distribute available devices among jobs without considering the specific resource scarcity or contention patterns. For instance, a job requiring widely available keyboard data might inadvertently consume devices that also possess scarce emoji data, thereby delaying other jobs that only require emoji data. This lack of awareness regarding resource scarcity and contention leads to suboptimal resource utilization and unnecessarily prolonged job completion times, motivating the need for a more sophisticated, contention-aware resource manager like Venn.

Key Findings

▶ Watch: Why existing single-job solutions fail for contention (4:00)

Venn's primary contribution is a novel resource manager that fundamentally rethinks how collaborative learning jobs are scheduled and resources are allocated on heterogeneous edge devices. The key findings and contributions presented in the talk are:

1. Identification of Complex Contention Patterns: The research highlights that simplistic resource allocation strategies, such as random device matching, fail to account for the varying scarcity of resources (e.g., specific data types or hardware capabilities) across different collaborative learning jobs. This oversight leads to significant inefficiencies and increased job completion times in multi-job environments.

1. Introduction of Contention-Aware Scheduling: Venn proposes a sophisticated scheduling algorithm that explicitly considers resource scarcity and job demands. It categorizes jobs into job groups based on shared resource requirements and then applies a two-tiered prioritization strategy:

• Across Job Groups: Prioritize groups that require the most scarce resources.
• Within Job Groups: Prioritize jobs based on their individual resource demand (e.g., smaller demand first).

This approach directly addresses the "scheduling delay" component of job completion time, which is ignored by single-job schedulers and mishandled by random allocation.

1. Significant Reduction in Job Completion Time: Through its contention-aware scheduling and resource-aware matching components, Venn demonstrates a substantial improvement in the overall job completion time for collaborative learning tasks. Experimental results show that Venn can achieve an impressive speedup of up to 1.87 times compared to baselines using random matching algorithms. This speedup encompasses both the reduction in scheduling delay and an optimized response collection time.

1. Faster Model Convergence: A direct consequence of reduced job completion times is faster model convergence. By minimizing the time each job spends waiting for or collecting resources, Venn allows individual models to complete their training iterations more quickly, leading to an accelerated overall convergence process for the learning tasks.

1. Theoretical Guarantees for Scheduling: While not detailed extensively in the talk, the speakers mention that the paper provides theoretical guarantees for the proposed scheduling problems, suggesting a robust foundation for Venn's algorithmic design. This indicates that the principles of prioritizing scarce resources and demand-based scheduling are not merely heuristic but grounded in formal analysis.

In essence, Venn's key findings underscore that intelligent, context-aware resource management is paramount for scaling collaborative learning. By proactively understanding and mitigating resource contention, Venn provides a framework that not only optimizes system throughput but also accelerates the development and deployment of personalized AI models.

Technical Deep Dive

▶ Watch: Venn: A contention-aware resource manager for ML jobs (8:00)

Venn's architecture is built around two core components designed to optimize the end-to-end job completion time for collaborative learning tasks: contention-aware scheduling and resource-aware matching. The talk primarily focuses on the former, which addresses the critical problem of scheduling delay – the time a job waits to acquire a sufficient number of eligible devices before training can commence.

Contention-Aware Scheduling

The fundamental insight behind Venn's scheduling is that not all resources are equally abundant, and different jobs have varying access patterns to these resources. Ignoring these complex contention patterns leads to suboptimal performance. To illustrate this, the speaker uses a compelling toy example:

• Device Pool: Imagine a pool of devices where 50% possess "emoji data" and 100% possess "keyboard data." This distribution is visualized over time, showing devices checking in, with emoji-capable devices being scarcer.
• Job Workload: Three jobs (J1, J2, J3) are introduced:
• J1: Requires "keyboard data" (can run on any device).
• J2, J3: Require "emoji data" (can only run on the 50% of devices with emoji data).
• J3 is assumed to have less resource demand than J2 (e.g., needs fewer devices or completes faster once started).

Baseline Approach (Smallest Remaining Job First - SRJF):

A common scheduling heuristic, SRJF, prioritizes jobs based purely on their estimated remaining demand. In this scenario, it might sort jobs as J1 > J3 > J2. When devices become available, SRJF would first allocate them to J1, then J3, then J2.

• Problem: J1, despite being able to run on any device, would receive priority and consume devices, including those that also have scarce emoji data. This effectively starves J2 and J3, which exclusively need emoji data, leading to a significantly prolonged scheduling delay for them. In the example, the total scheduling delay for all jobs was 29 units. This highlights how ignoring resource scarcity can lead to inefficient resource allocation and increased overall completion times.

Venn's Contention-Aware Scheduling Principles:

Venn's approach is guided by two core principles that collectively form a more intelligent prioritization strategy:

1. Prioritize Scarce Resources Across Job Groups:

• Venn first identifies job groups—sets of jobs that share the same resource requirements. In the example, J2 and J3 form an "emoji data" job group, while J1 forms a "keyboard data" job group.
• The principle dictates that job groups requiring more scarce resources should be prioritized. Since emoji data is scarcer (50% availability) than keyboard data (100% availability), the emoji job group (J2, J3) is prioritized over the keyboard job group (J1).

1. Prioritize by Resource Demand Within Job Groups:

• Once a job group is prioritized, jobs within that group are then prioritized based on their individual resource demand. The talk implies a "smallest demand first" heuristic.
• In the emoji job group, J3 has less demand than J2. Therefore, J3 is prioritized over J2.

Combined Scheduling Decision:

Applying these principles, Venn derives a scheduling order: J3 > J2 > J1.

• Outcome: When devices become available, Venn first satisfies J3, then J2, and finally J1. This ensures that the scarce emoji-capable devices are preferentially allocated to the jobs that critically depend on them.
• Improved Performance: In the toy example, this strategy reduced the total scheduling delay from 29 units (SRJF) to 24 units, demonstrating the effectiveness of scarcity-aware prioritization.

The speaker notes that the paper provides "more theoretical guarantees" on these scheduling problems and considers "more complex resource contention that happens in the real world." This suggests a robust theoretical underpinning for Venn's heuristic principles, addressing scenarios beyond simple data type scarcity, potentially including multi-dimensional resource constraints (CPU, memory, specific software versions, etc.).

Resource-Aware Matching

While the talk primarily elaborates on contention-aware scheduling, Venn also includes a resource-aware matching solution. This component is designed to minimize the response collection time, which is the duration it takes for a job to gather enough device responses after it has been scheduled and started. Although not detailed in the presentation, this component likely involves intelligently matching already-scheduled jobs with available devices based on fine-grained device characteristics and job requirements, ensuring that devices are not just available but also optimally suited for the assigned task, further accelerating the training process.

In summary, Venn's technical prowess lies in its departure from simplistic resource management. By explicitly modeling and addressing the intricacies of resource scarcity and job-specific demands on heterogeneous edge devices, it provides a principled framework for coordinating multiple collaborative learning jobs more efficiently.

Experimental Setup & Results

▶ Watch: Toy example: Complex contention patterns between jobs (10:00)

To validate Venn's effectiveness, the researchers conducted experiments designed to faithfully emulate real-world scenarios in collaborative learning environments. The setup emphasized using realistic device traces and diverse job types to create complex resource contention patterns.

Experimental Setup

1. Realistic Device Traces:

• Device Availability: The experiments incorporated real-world device availability traces, which showed patterns like increased device availability during nighttime hours, reflecting typical user behavior.
• Device Available Duration: Traces also captured how long devices remained eligible for training, accounting for their intermittent nature (e.g., connected to Wi-Fi, charging).
• Hardware Heterogeneity: A density map was used to visualize the distribution of device capabilities, specifically memory and CPU, across the device pool. This map, derived from real traces, demonstrated the significant heterogeneity in edge device hardware, which is a critical factor in collaborative learning performance.

1. Diverse Job Workloads:

• Four Types of Jobs: To simulate varied resource requirements and induce contention, the experiments defined four distinct job types:
• Jobs eligible for any device.
• Jobs eligible only for memory-rich resources.
• Jobs eligible only for compute-rich resources.
• Jobs requiring both memory-rich and compute-rich resources.
• Resource Demand: The resource demand for each job (e.g., number of devices needed, specific hardware minimums) was sampled from distributions extracted from real-world federated learning papers from industry, ensuring the workload reflected practical collaborative learning scenarios.

1. Workload Generation: The researchers created different workloads to simulate various resource contention patterns. This allowed for a comprehensive evaluation of Venn's performance under a range of scarcity and demand scenarios.

Results

The primary metric for evaluating Venn's performance was the job completion time (JCT), which encompasses both the scheduling delay and the response collection time. The results were reported as improvements over a random matching algorithm baseline, a common, albeit oversimplified, approach in existing multi-job collaborative learning systems.

1. Significant Speedup in Job Completion Time:

• Venn consistently outperformed the random matching baseline across different workloads and contention patterns.
• The headline result demonstrated an impressive speedup of up to 1.87 times in job completion time. This significant improvement underscores the effectiveness of Venn's contention-aware scheduling and resource-aware matching in mitigating delays and optimizing resource utilization.

1. Faster Model Convergence:

• Beyond just reducing completion time, the experiments also tracked aggregated model convergence curves.
• The results showed that because each job completed faster under Venn's management, the individual models were able to converge more quickly. This has direct implications for accelerating the development and deployment cycles of collaborative learning models.

The evaluation process, anchored by realistic device traces and industry-derived job characteristics, provides strong evidence that Venn's principled approach to resource management offers substantial benefits for large-scale, multi-job collaborative learning deployments. The observed speedup and faster convergence rates are critical for enhancing the efficiency and practical utility of federated learning systems.

Practical Implications

▶ Watch: Venn's scarcity-aware scheduling compared to baseline (12:00)

Venn's resource management framework has profound practical implications for various stakeholders involved in the development and deployment of collaborative learning systems, particularly in large-scale enterprise environments.

For Practitioners and Infrastructure Teams:

• Optimized Resource Utilization: Venn provides a blueprint for infrastructure teams to more efficiently utilize scarce and heterogeneous edge device resources. By moving beyond random allocation, it ensures that devices are not just available but are matched to jobs that truly need their specific capabilities or data, preventing underutilization or resource starvation.
• Reduced Operational Costs: Faster job completion times mean that computational resources (both on-device and server-side for aggregation) are engaged for shorter durations per task. This can translate into reduced operational costs, especially in cloud-hosted aggregation environments or when considering device battery life.
• Improved System Throughput: The ability to manage complex contention patterns allows for a higher number of concurrent collaborative learning jobs to run effectively. This increases the overall throughput of the system, enabling organizations to experiment with more models or deploy more personalized services simultaneously.
• Predictable Performance: By actively managing scheduling delays and resource contention, Venn can help introduce more predictability into collaborative learning job execution times. This is crucial for planning, setting expectations, and ensuring service level objectives (SLOs) are met, especially for time-sensitive model updates.

For Model Builders and Deployers:

• Accelerated Model Development: Model developers can iterate faster on new models or improvements. Reduced job completion times mean quicker experimentation cycles, allowing for more rapid testing of different architectures, hyperparameters, or data strategies.
• Enhanced Personalization: By efficiently managing resources for jobs requiring specific, often scarce, user data (e.g., emoji usage patterns), Venn facilitates the development of highly personalized models that are truly tailored to individual user behaviors and preferences.
• Broader Application Scope: The ability to handle diverse job requirements and device heterogeneity means that collaborative learning can be applied to a wider range of use cases, including those with stringent resource constraints or highly specialized data needs.

Tradeoffs and Limitations:

While Venn offers significant advantages, its practical implementation comes with certain considerations:

• Complexity of Implementation: Implementing a contention-aware scheduler requires a sophisticated understanding of device capabilities, real-time device availability, and the precise resource requirements of each job. This necessitates robust device profiling, metadata management, and a centralized scheduling component, which can be complex to build and maintain.
• Data Collection and Analysis: Accurately identifying "scarce resources" and predicting "resource demand" requires collecting and analyzing extensive real-world traces of device availability, hardware specifications, and job characteristics. The quality of these inputs directly impacts the scheduler's effectiveness.
• Focus on Scheduling Delay: The talk primarily deep-dived into contention-aware scheduling for reducing scheduling delay. While resource-aware matching for response collection time was mentioned, its detailed mechanisms and practical implications were not fully explored. A complete solution would require both components to be equally robust.
• Real-World Edge Deployment: While the experiments used realistic traces, deploying such a system on truly "wild" edge devices introduces further challenges related to network connectivity, device churn, and user privacy controls, which may not be fully captured in simulations or server-side emulations.
• Overhead of Resource Awareness: Maintaining detailed information about device capabilities and job requirements, and running a more complex scheduling algorithm, might introduce some computational or communication overhead compared to simpler random matching. However, the demonstrated speedups suggest this overhead is far outweighed by the benefits.

In conclusion, Venn provides a critical step forward for making collaborative learning truly scalable and efficient in multi-tenant, heterogeneous edge environments. Its principles can guide the design of next-generation resource managers, enabling faster, more personalized, and cost-effective AI development at the edge.

Key Takeaways

• Collaborative learning on heterogeneous edge devices faces significant resource management challenges, particularly with multiple concurrent jobs.
• Existing solutions are insufficient, either focusing only on single jobs or using oversimplified random device matching, leading to substantial "scheduling delays" and inefficient resource utilization.
• Venn introduces a novel contention-aware scheduling framework that explicitly considers resource scarcity and job-specific demands.
• Its core principles involve prioritizing jobs requiring scarcer resources across job groups and then prioritizing by resource demand within those groups.
• Venn achieves up to a 1.87x speedup in job completion time compared to random matching baselines, significantly reducing delays.
• Faster job completion directly leads to accelerated model convergence, enabling quicker iteration and deployment of personalized AI models.
• This approach has significant practical implications for optimizing resource utilization, reducing operational costs, and accelerating AI development in large-scale collaborative learning deployments.

About the Speaker(s)

The talk was presented by Jiachen Liu, who is from the University of Michigan. He is one of the co-authors of the research paper on Venn, alongside Fan Lai, Eric Ding, Yiwen Zhang, and Mosharaf Chowdhury. Their work focuses on resource management for collaborative learning jobs, aiming to improve efficiency and speed up training processes in distributed, heterogeneous environments.