Download: PDF.
“Experimenting with software testbeds for evaluating new technologies” by Mikael Lindvall, Ioana Rus, Paolo Donzelli. Atif Memon, Marvin Zelkowitz, Aysu Betin-Can, Tevfik Bultan, Chris Ackermann, Bettina Anders, Sima Asgari, Victor Basili, Lorin Hochstein, Jörg Fellmann, Forrest Shull, Roseanne Tvedt, Daniel Pech, and Daniel Hirschbach, Empirical Software Engineering: An International Journal, vol. 12, no. 4, 2007, pp. 417-444, Kluwer Academic Publishers.
The evolution of a new technology depends upon a good theoretical basis for developing the technology, as well as upon its experimental validation. In order to provide for this experimentation, we have investigated the creation of a software testbed and the feasibility of using the same testbed for experimentingwith a broad set of technologies. The testbed is a set of programs, data, and supporting documentation that allows researchers to test their new technology on a standard software platform. An important component of this testbed is the Unified Model of Dependability (UMD), which was used to elicit dependability requirements for the testbed software. With a collection of seeded faults and known issues of the target system, we are able to determine if a new technology is adept at uncovering defects or providing other aids proposed by its developers. In this paper, we present the Tactical Separation Assisted Flight Environment (TSAFE) testbed environment for which we modeled and evaluated dependability requirements and defined faults to be seeded for experimentation. We describe two completed experiments that we conducted on the testbed. The first experiment studies a technology that identifies architectural violations and evaluates its ability to detect the violations. The second experiment studies model checking as part of design for verification.We conclude by describing ongoing experimental work studying testing, using the same testbed. Our conclusion is that even though these three experiments are very different in terms of the studied technology, using and re-using the same testbed is beneficial and cost effective.
Download: PDF.
BibTeX entry:
@article{LindvallEMSE2007,
author = {Mikael Lindvall and Ioana Rus and Paolo Donzelli and Atif
Memon and Marvin Zelkowitz and Aysu Betin-Can and Tevfik Bultan
and Chris Ackermann and Bettina Anders and Sima Asgari and Victor
Basili and Lorin Hochstein and J{\"o}rg Fellmann and Forrest Shull
and Roseanne Tvedt and Daniel Pech and Daniel Hirschbach},
title = {Experimenting with software testbeds for evaluating new
technologies},
journal = {Empirical Software Engineering: An International Journal},
volume = {12},
number = {4},
pages = {417--444},
publisher = {Kluwer Academic Publishers},
address = {Hingham, MA, USA},
year = {2007},
issn = {1382-3256}
}
Back to Atif Memon's Publications.