*Q1*: Do you think that using just screenshots, instead of live systems
affect the evaluation result or not? Why and in which cases might it still
be useful?

*A*: Using any kind of prototype to test the usability of a system may
influence the results of usability testing. Whereas on the one hand
prototypes like screenshots make it possible to evaluate the user interface
already during the development process and allow to make necessary changes
on the fly (and therefore less expensive) they are possibly not implementing
all the features the later system is going to have. As for screenshots it
might give the user an already good impression how static parts of the
systems are going to look like and help to reveal certain kinds (or a subset
of all final) usability faults (e.g. use of icons and terms, basic
workflow). Summary: yes, it affects the evaluation result but is useful to
do during the development process.


*Q2:* A usability fault is not as well-defined as a correctness fault.
Imagine that, in a group of testers, some say that a particular feature of
an application is a usability fault, but others like that feature the way it
is and say it should not be changed. How would you decide whether this is a
fault or not?

*A*: In such cases it is important to know the users of your system and to
compare them with your usability testers. In case it turns out that one of
two arguing test groups represents the application users then take their
advice. In cases where no decision can be made it might be possible to let
the user choose between the possibility to activate or deactivate the
feature. Alternatively it might be possible to modify the feature in a way
that both groups are satisfied. If none of this works and the feature
significantly prevents the one group of users to fulfill a common task it
might be the right thing to consider it as usability fault.

*Q3*: What is "aggregated evaluators"? Give the motivation and how to do
that.

*A*: In usability testing aggregation of evaluators is a technique used in
heuristic evaluation to increase the number of found usability faults. Basic
idea is that the aggregation of faults found by a group of evaluators covers
the set of usability faults very well: depending on application 80% of the
faults are already found by 5 to 10 evaluators. The process to execute is to
let each evaluator find independently a set of usability faults by giving
them some set of basic guidelines (heuristics) and then to build the total
set of found faults by combining these sets.