JMS: Joint Bandwidth Allocation and Flow Assignment for Data Transfers with Multiple Sources

Abstract The increasing prevalence of data-intensive applications has made large-scale data transfers more important in datacenter networks. Oversubscribed networks have excessive traffic demand causing serious performance bottlenecks. Data replicas, with the advantage of source diversity, can potentially improve the transmission performance, but current works focus heavily on best replica selection rather than multi-source transmission. In this paper, we present JMS, a novel traffic management system that optimizes bulk multi-source transfers in software-defined datacenter networks. With a global network view and consistent data access, JMS conveys data in parallel from multiple distributed sources and dynamically adjusts the flow volumes to maximize network utilization. The joint bandwidth allocation and flow assignment optimization problem poses a major challenge with respect to nonlinearity and multiple objectives. To cope with this, we design an online fair allocation algorithm that derives a novel transformation with simple equivalent canonical linear programming to achieve global optimality efficiently. Simulation results demonstrate that JMS outperforms other single-source and multi-source transmission approaches with substantial gains, where JMS improves the network throughput by up to 52% and reduces the transfer completion time by up to 44%, meanwhile it continues to provide good performance for large files as well as high traffic loads.
Authors
  • Geng Li (Yale)
  • Yichen Qian
  • Lili Liu
  • Richard Yang (Yale)
Date Jul-2018
Venue 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC)