Andreas Birk, AG Software Engineering
Department of Computer Science, University of Kaiserslautern
D-67653 Kaiserslautern, Germany, e-mail: birk@informatik.uni-kl.de
Important tasks for the software engineering discipline are to experiment with software engineering technologies, to find out how and when they work well, and to understand how to improve them [Bas95]. Performing these tasks requires appropriate concepts for experimentation as well as a means for storing, maintaining, and making accessible the experiences gained through experimentation. ESPRIT project PERFECT aims at assisting industry in systematic improvement of quality in software projects. Therefore, a set of technologies has been chosen that is based on the NASA/SEL approach for continuous improvement. Among these technologies are the Quality Improvement Paradigm (QIP), the Experience Factory (EF) concept [BCR94a], and the Goal / Question / Metric (GQM) approach [BW84], [BCR94b]. A major task for PERFECT is to collect and package experiences with these improvement technologies and to make them usable for a broad spectrum of organizations. Special attention is paid to gaining and documenting experiences with GQM-based measurement programs.
For facilitating the reuse of technologies and for supporting their evolution and improvement, it is suggested to document technologies using the following conceptual structure [PERF95]:
- Goals of the technology
- Principles of the technology
- Method that represents the technology
- Techniques for supporting the method
- Tools for supporting the techniques
- Applications of the method, its techniques, and its tools
The goals make explicit what the technology is for, and the principles provide an abstract specification of the technology. A method consists of a set of techniques which are associated with guidelines about when and how to apply the techniques [BCR94a]. Methods can be described using process models. Tools can be viewed as implementations of techniques.
Documentation of technologies should also involve application reports that allow for understanding the technology better, and for deriving experiences with the method, its techniques, and its tools. Application experiences are a basis for improving a technology and for justifying modifications.
In order to understand whether a technology has been useful for a particular task, it should be documented together with the application context (project and organization) and a description of how it has been adapted to this context. Hence, documentation of application experiences should contain the following information:
- Characterization of the organization
- Characterization of the project
- Technology baseline
- Technology instantiation
- Goals of using the technology
- Which parts of the method have been altered, omitted, or supplied?
- Experiences
- Problems and assumed causes (from the project perspective)
- Benefits and assumed causes (from the project perspective)
- Experiences from experimenting with the technology (from the technology perspective)
Context characterizations should provide all information that is necessary to understand why the technology has been used, why it has been instantiated in a specific way, and why each particular experience is viewed as relevant. Technologies are always described at an abstract level, independent from a particular application context. In order to draw conclusions from an application, it is necessary to understand how the technology has been adapted to the project. A technology baseline describes which description or variant of the technology is being used in the application. A technology instantiation describes why this technology has been chosen, and how it has been adapted.
There are two kinds of perspectives for which experiences can be relevant. Project managers and project team members ("project perspective") want to apply a technology as a means to an end. Usually, they can report problems that occurred during application and benefits they perceived. Problems and benefits should always be documented together with their assumed causes in order to provide a starting point for further investigations. The second perspective is interested in evaluating and improving a technology ("technology perspective"), so questions may be interesting that are beyond the usual scope of the project perspective. They can be investigated by using the application project as an experiment [Bas92]. All data and information that is gained by those experiments should be documented together with all other application information in order to achieve a complete picture of what is known about a technology.
Currently, the concepts described above are being evaluated in project PERFECT. There are ongoing activities of capturing application experiences on performing GQM-based measurement programs and other aspects of improvement programs. Possibly, some of these experiences will be reused in some PERFECT applications before the end of the project in autumn 1996. It is expected that these activities will provide a better understanding of how to gather and represent application experiences, of how to characterize projects and organizations, of the usability of this approach for documenting application experiences, and of the effort needed for it.
[Bas92]
Victor R. Basili. The experimental paradigm in software engineering. In H. D. Rombach, V. R. Basili, and R. W. Selby, editors, Experimental Software Engineering Issues: A critical assessment and future directions, pages 3-12. Lecture Notes in Computer Science Nr. 706, Springer-Verlag, September 1992.
[Bas95]
Victor R. Basili. The Experience Factory and its relationship to other quality approaches. In Marvin V. Zelkowitz, editor, Advances in Computers, vol. 41, pages 65-82. Academic Press, 1995.
[BCR94a]
Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. Experience Factory. In John J. Marciniak, editor, Encyclopedia of Software Engineering, volume 1, pages 469-476. John Wiley & Sons, 1994.
[BCR94b]
Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. Goal Question Metric Paradigm. In John J. Marciniak, editor, Encyclopedia of Software Engineering, volume 1, pages 528-532. John Wiley & Sons, 1994.
[BW84]
Victor R. Basili and David M. Weiss. A methodology for collecting valid software engineering data. IEEE Transactions on Software Engineering, SE-10(6):728-738, November 1984.
[PERF95]
PERFECT Consortium. The PERFECT approach to continuous improvement using experience. Proceedings of ESI-ISCN'95: Measurement and Training Based Process Improvement. Vienna, Austria, 1995.