Background and Motivation
The deployment of artificial intelligence workloads at the edge has become critical for applications requiring low latency, privacy preservation, and bandwidth optimization. Modern AI applications rely on complex multi-stage pipelines that include data preprocessing, model inference, post-processing, and result aggregation. However, managing these pipelines across distributed edge infrastructures presents significant orchestration challenges due to resource heterogeneity, network variability, and dynamic workload demands.
Traditional cloud-based orchestration platforms like Kubernetes are ill-suited for edge environments due to their assumptions about reliable connectivity, homogeneous hardware, and centralized control. Oakestra represents a breakthrough in edge orchestration, providing a lightweight, hierarchical framework specifically designed for heterogeneous edge computing environments. Existing state-of-the-art orchestration frameworks (e.g. Kubernetes) perform poorly at the edge since they were designed for reliable, low latency, high bandwidth cloud environments.
AI pipeline management at the edge introduces unique challenges including dynamic model partitioning across devices with varying computational capabilities, intelligent scheduling of pipeline stages to minimize end-to-end latency, and efficient resource allocation considering both computational and communication costs. Current MLOps practices focus primarily on cloud deployments and lack the sophistication needed for distributed edge AI workloads where network topology, device capabilities, and data locality significantly impact performance.
Expected Outcomes
This thesis will develop an intelligent AI pipeline orchestration system integrated with Oakestra's hierarchical architecture to optimize the deployment and execution of machine learning workloads across distributed edge infrastructures. The research will create adaptive scheduling algorithms that dynamically partition AI pipelines based on real-time resource availability, network conditions, and application requirements.
The core contribution will be a pipeline-aware orchestration framework that automatically decomposes AI workflows into optimal stages for distributed execution. The system will implement intelligent placement policies that consider data locality, model complexity, and inter-stage communication overhead. Dynamic load balancing mechanisms will redistribute pipeline stages across edge nodes to maintain performance under varying conditions.
Integration with Oakestra's existing infrastructure will enable seamless deployment of AI pipelines across multi-tier edge architectures. The system will provide automated model optimization including quantization, pruning, and compilation for target hardware platforms. Pipeline monitoring and profiling capabilities will enable continuous optimization based on observed performance metrics and resource utilization patterns.
Experimental evaluation will demonstrate the framework's effectiveness using real-world AI applications including computer vision, natural language processing, and IoT analytics workloads. Performance metrics will include end-to-end pipeline latency, resource utilization efficiency, and scalability across varying edge infrastructure sizes. The research will establish best practices for AI pipeline orchestration in edge environments and provide guidelines for application developers deploying distributed AI workloads.
Requirements
• Strong understanding of machine learning pipeline architectures and MLOps practices • Experience with container orchestration systems and distributed computing frameworks • Proficiency in Python and Go for orchestration system development • Knowledge of edge computing architectures and resource-constrained environments • Familiarity with AI/ML frameworks (TensorFlow, PyTorch) and model optimization techniques
Related Work
[1] Bartolomeo, G., Yosofie, M., Bäurle, S., Haluszczynski, O., Mohan, N., & Ott, J. (2023). Oakestra: A lightweight hierarchical orchestration framework for edge computing. In 2023 USENIX Annual Technical Conference (USENIX ATC 23) (pp. 215-231).
[2] Yuanming Shi, Kai Yang, Tao Jiang, Jiangtian Nie, and Shi Jin. 2020. Communication-Efficient Edge AI: Algorithms and Systems. IEEE Communications Surveys & Tutorials 22, 4 (2020), 2167–2191. https://doi.org/10.1109/COMST.2020.3007787
