Inter-Datacenter Bulk Transfers with NetStitcher
We are conducting research on how to rescue unutilized bandwidth across multiple datacenters and backbone networks.
Large datacenter operators with sites at multiple locations design their inter-datacenter networks such that they can accommodate the maximum bandwidth needed during a day. At the same time, the demand for inter-datacenter bandwidth follows strong diurnal patterns with high peak to valley (i.e. high to low) ratios that result in poor average utilization across a day.
We are conducting research on how to rescue unutilized bandwidth across multiple datacenters and backbone networks. The rescued bandwidth can be used by non-real-time applications, such as backups, propagation of bulky updates, and migration of data and/or virtual machines that improve fault tolerance, end-user experience, and energy/ personnel costs, respectively. Achieving the above is non-trivial since leftover bandwidth appears at different times, for different durations, and at different places in the world.
For this purpose, we have designed, implemented, and validated NetStitcher, a system that employs a network of storage nodes to stitch together unutilized bandwidth, whenever and wherever it exists. Our system gathers information about leftover resources, uses a store-and- forward algorithm to schedule data transfers, and adapts to resource fluctuations.
We have compared NetStitcher with other bulk transfer mechanisms such as direct transfer, multipath forwarding, and naive store-and-forward. Our evaluation showsthat NetStitcher outperforms all other mechanisms and can rescue up to five times additional datacenter bandwidth thus making it a valuable tool for datacenter providers.
• Good Things Come to Those Who (Can) Wait or How to Handle Delay Tolerant Traffic and Make Peace on the Internet, N. Laoutaris, P. Rodriguez, ACM HotNets'08.
We have deployed NetStitcher on Patents:
Telefonica's Global Content Distribution Network to demonstrate that our solution can perform large data transfers at a much lower cost than naive end-to-end or store- and-forward schemes.
"Our system gathers information about leftover resources..."