Preparations, changes from previous execution

Generic documents:

The generic documents were reviewed. First, the ABC video system and the ATM network documents were reviewed with ad hoc techniques. Then, these two documents were discussed, and some changes made. The Parking Garage document was reviewed after a session of training in perspective-based reading. The discussion after the reviews brought up a few changes to the defect list as well as the document itself. In particular, the introduction and the general description were made error free. However, the amount and type of the changes were limited by the fact that we wanted to keep a basis of comparison between the results from each execution of the experiment.

NASA documents:

These were made shorter than the previous time. A short introduction was added to make these documents more similar to the generic ones. The difference in nature between the NASA documents and the generic documents are expected to be smaller this time.

Forms:

The forms were discussed and found to be basically OK. Some language concerns were straightened out, especially on the defect classification (which was confusing the way it was). Form E7 was enhanced to remove the need for the interviews. Form E6 for the user perspective was changed to accomodate situations where the user role is less obvious. Also, the view of this perspective as 'writing a user's manual' was removed. The perspective is now called 'use-based'. The questions on the scenarios were discussed, and some questions were found to be redundant.

Training:

This time, training was provided for the NASA documents specificially (see schedule below). We had one general introduction to the entire experiment, the forms, the error classification etc. Then, for each pair of document type and technique type (four pairs) we had a preceding training session made particularly for this combination of technique and document.

In the first session, the participants were informed that the introduction and general description of all the documents were (assumed to be) error free.

Roles:

We wanted to get as many people as possible for this replication. This forced us to not consider matching the different perspectives to the most relevant people, because this would require a uniform distribution of people over the different roles. Also, an arbitrary relationship between people and roles would make the experiment randomized rather than blocked.

Schedule:

All participants followed the same schedule on the first day. It was announced that there would be coffe and donuts between 9 and 9:30 to ensure that the participants arrived in time. The time frame was 9:30 to 3 and allowed for 15 minutes delay. In the schedules below, we used numbers 1, 2 and 3 for the perspectives design, test and use respectively. The reason for this was just to avoid any interference between the perspectives.

On the second day, the participants were split up in three groups according to their assigned perspective. This day also the time frames had room for 15 minutes of delay. The different groups had the following schedules:

Web Accessibility