2. Different Types of Benchmarks

2.1 Levels and Types of Benchmarks

Based on the levels of performance they measure, benchmarks can be grouped into two levels: Based on their compositions, benchmarks can be categorized into types:

2.2 Component-Level Benchmarks

Component-Level benchmarks test a specific component of a computer system, such as the video board, the audio card, or the microprocessor. They are useful for selecting a component of a computer system, that corresponds to a particular function. Instead of testing the performance of the system running real applications, component-level benchmarks focus on the performance of subsystems within a system. These subsystems may include operating system, arithmetic interger unit, arithmetic floating-point unit, memory system, disk subsystem, etc. Examples of component-level benchmarks include:

2.3 System-Level Benchmarks

System-Level benchmarks evaluate the overall performance of a computer running real programs or applications. These benchmarks are useful when comparing systems of different architectures. They take each subsystem into account, and indicate the effect of each subsystem on the overall performance. Examples of system-level benchmarks include:

2.4 Synthetic Benchmarks

Synthetic benchmarks are created by combining basic computer functions in proportions that developers feel will yield an indicative measure of the performance capabilities of the machine under test. These benchmarks try to match the average frequency of operations and operands of a large set of programs. Functions included in synthetic benchmarks are usually created artificially to match an average execution profile and this impairs their credibility.

Synthetic benchmarks are component-level benchmarks, and they evaluate a particular capability of a subsystem. For example, a disk subsystem performance benchmark may combine a series of basic seek, read, and write operations involving varying numbers of disk blocks of varying sizes.

When evaluating the results from synthetic benchmarks, the following rules should be followed:

Examples of synthetic benchmarks include:

2.5 Application Benchmarks

Application benchmarks employ actual application programs. Developers of these benchmarks include applications that they feel perform common functions from within a particular industry segment or a class of products. These application programs are run by a macro of program operations which attempts to model the way users operate their system.

Most application benchmarks are system-level benchmarks, and they measure the overall performance of a system. When an application benchmark is run, it tests the contribution of each component of the system to the overall performance. These benchmarks are usually larger and difficult to execute and are not useful for measuring future needs.

The major drawback is that application benchmarks are subject to the benchmark developer's interpretation of a "typical workload". Examples of application benchmarks include:

| Overview | Introduction | Engineering | Reporting | Revising |
|Questions |