By Jyrki Kontio
In total we received 21 questionnaires. Most organizations were able to offer their data widely available, only three organizations limited the data to summaries (question 1). To include these organizations in this data summary and to keep the report within reasonable size, we have reported the data on summary level for all organizations. While this will probably give a suitable overview of the data for most people, the summary level data does not support analysis based on blocking factors or making cross references between items.
Most organizations in the survey were large, 60% having over 5,000 employees (question 4). Most of them (66%) also had also a quarter or more of their personnel involved in software development (question 5 and question 5). Most organizations were located in one or two sites (question 10).
Project sizes ranged from about a person month to 600 person years, typical size being ten person years (question 11). Variations in duration were smaller, typical duration being 1-2 years (question 12). Initial deliveries or new versions of systems seemed to be the most typical project type (question 13).
Subcontracting seemed to be common: a quarter subcontracted more than developed in-house (question 8).
Of those who responded, 63% had gone through a formal assessment and additional 16% had performed informal assessments (question 14). Over half had been exposed to ISO 9001 and 40% to CMM (question 15). Nine organizations claimed to have an ISO 9001 certificate, five thought of themselves as CMM level 3 organizations (question 16).
Almost all organizations had explicit software process definitions (question 18) but they were usually not strongly enforced (question 21). Project management cost estimation tools were the most widely used tools for management (question 22). Over half had a dedicated group for process management (question 26).
Independent testing was used only by nine out of 19 organizations (question 24), where as two thirds had independent quality assurance function (question 25).
Data collection was common and started early in all organizations (question 27) which is not surprising given the topic of the workshop. Most organizations seemed to continue data collection well after the initial delivery: some (33%) continued as long as the system was in use. (question 28).
A third used an integrated database for storing the date but almost 40% relied on manual methods (question 30).
The two most common application types were "embedded/real-time systems" and "transaction processing" (question 31). Telecommunications stood out as the main application domain (question 32). Unique software development and embedded software were the two most common product types (question 33).
The most important product and process characteristics were product quality, product functionality and cycle time (question 34). Productivity, profit and consistency were clearly ranked lower in importance. Reliability and functionality were the most important quality characteristics for most organizations (question 35). Project schedule and product quality were considered the most important project management constraints (question 36).
Project managers and personnel, as well as QA group and organization management were the most frequent users of defect data. Eight organizations (38%) allowed customers to access defect data (question 37).
The defect data was primarily used for determining acceptance of software (86%), project status monitoring (71%) and identifying areas for process improvement (67%) (question 40).
Most organizations rely on using simple statistical techniques or graphics in analyzing defect data. Less than third used either structured approaches (such as GQM) or statistical techniques (question 41).
Cost data is primarily used for controlling project progress (question 44). Organizations used various approaches in consolidating the data. Most common aggregation principles were calendar time, process phase, and organizational structure (question 45).
Risk management questions were included in the questionnaire for the first time. Twelve organizations from our total of 21 reported to have some experience in risk management techniques, although only four reported to have used them "extensively" (question 49). Techniques seemed to be in broad use in most organizations (question 53). Most common risks seemed to involve schedule and product quality impacts (question 48).
The organizations that had used risk management techniques were not enforcing or supporting them strongly (question 50) but most (75%) required some risk analysis to take place frequently (question 51).
Responses to the questions related to monitoring of risk management activity (question 54-56) indicated that organizations rely on gut feelings on the effectiveness and cost of these methods.
The risk management techniques that were used were mainly simple ones, data analysis, checklists, and brainstorming (question 59). The expected value of risk impact (i.e., risk exposure) was used only by 25% of the respondents (question 60).
As far as risk reduction is concerned, changing goals or relaxing constraints seemed to be the primary courses of action (question 61).
Many organizations indicated an interest to deploy or try out risk management methods (question 57). It seems that a major reason for low usage of explicit risk management methods is lack of knowledge about them, not necessarily any knowledge of their ineffectiveness (question 58).
Updated 07-Feb-96 by Jyrki Kontio