att_abstract={{Cellular network service providers often have to conduct small scale testing in the operational network before a change (e.g., a new feature) is fully rolled out across the entire network. This is referred to as the First Field Application (FFA). However, assessing the effectiveness of FFA changes is challenging because of overlapping external factors: seasonality (foliage, leaves budding), weather (rain, snow, hurricanes, storms), traffic pattern changes due to big events (e.g., games at stadiums, students returning to school after holidays), and network events such as outages or other maintenance activities in different regions. In this paper, we first highlight the technical challenges in assessing the service performance impact of changes in operational cellular networks. We then propose Litmus, a new approach based on a spatial dependency model for robust assessment of changes. We evaluate the effectiveness of Litmus using real-world data from operational cellular networks (GSM, UMTS and LTE). Our operational experiences demonstrate accurate inferences of the service performance impact of changes in the field.
	att_authors={am649n, zg2325, jy1348, hh6807, vc9984, cs4017, xj6779, ms929f},
	att_copyright_notice={{(c) ACM, 2013. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in 2013 {{, 2013-12-09}}.
	author={Ajay Mahimkar and Zihui Ge and Jennifer Yates and Hristo Hristov and Vincent Cordaro and C Smith and Jing Xu and Mark Stockert},
	institution={{ACM CoNEXT 2013}},
	title={{Robust Assessment of Changes in Cellular Networks }},