Effective and Scalable Software Compatibility Testing

Download: PDF.

“Effective and Scalable Software Compatibility Testing” by Il-Chul Yoon, Alan Sussman. Atif M. Memon, and Adam Porter, in ISSTA '08: Proceedings of the International Symposium on Software Testing and Analysis, (Washington DC, USA), 2008.

Abstract

Today's software systems are typically composed of multiple components, each with different versions. Software compatibility testing is a quality assurance task aimed at ensuring that multi-component based systems build and/or execute correctly across all their versions' combinations, or configurations. Because there are complex and changing interdependencies between components and their versions, and because there are such a large number of configurations, it is generally infeasible to test all potential configurations. Consequently, in practice, compatibility testing examines only a handful of default or popular configurations to detect problems; as a result costly errors can and do escape to the field. This paper presents a new approach to compatibility testing, called Rachet. We formally model the entire space of configurations for component-based systems and use the model to generate test plans to sample a portion of the space. In this paper, we cover all direct dependencies between components. The test plan is executed efficiently in parallel. We present experiments and simulation studies applying our approach to two large-scale scientific middleware systems. The results show that for these two systems Rachet scaled well and discovered incompatibilities between components, and that testing only direct dependences did not compromise test quality.

Download: PDF.

BibTeX entry:

@inproceedings{YoonSussmanMemonPorterISSTA2008,
   author = {Il-Chul Yoon and Alan Sussman and Atif M. Memon and Adam Porter},
   title = {Effective and Scalable Software Compatibility Testing},
   booktitle = {ISSTA '08: Proceedings of the International Symposium on
	Software Testing and Analysis},
   publisher = {IEEE Computer Society},
   address = {Washington DC, USA},
   year = {2008}
}

Back to Atif Memon's Publications.