1. Introduction

1.1 The Role of Benchmarks in Computer Architecture
Benchmarks are individual programs or a mixture of programs that are run on a target computer to measure the overall performance of the system, or to measure more specific aspects of the performance, such as Graphics Applications, I/O Processing, Net Browsing, etc. Any aspects of computer performance that matters to the user can be benchmarked.
The role of benchmarks in computer architecture is to evaluate the performance of a particular system architecture and extrapolate from the results obtained. Benchmarks are used not only to evaluate the performance of a given system under different hardware and software configurations, but also to compare the performance of different systems. The performance of different functionalities of a machine can be predicted by running a representative workload on the machine. Some benchmarks test how fast a computer can perform a specific task whereas some test how fast the computer handles different tasks at the same time. For example, SPECint95 benchmark measures the performance of the processor only, but SYSmark/NT benchmark measures the overall performance of your entire PC when running real applications.
Nowadays, the role of benchmarks has been extended to help a computer user make critical decisions during the purchasing process. Benchmarks are essential to truly assess processing performance of a component or a system that the user considers purchasing. Benchmarks can be used as diagnostic tools to uncover the cause of poor performance of a configuration by comparing benchmark results against a known configuration. The potential impact of a system update on the overall performance can be predicted before actually implementing the update by the use of benchmarks.

1.2 Purpose of the Project
This project is intended to give the reader conceptual understanding on

what benchmarks are,
how they are used,
what kinds of performance they measure,
what the most common and popular benchmarks available on market are,
what their limitations and constraints are,
how can one safely interpret benchmark results, and
what the pitfalls and fallacies involved with running benchmarks.

1.3 Intended Audience
The primary audience of this document will be our CMSC 411 instructor, Dr. Michelle Hugue, and the secondary audience may include future CMSC 411 students. The audience is expected to:

have prior experience with basic computer programming,
understand the organization of a computer,
have read the first chapter of the prescribed textbook for CMSC 411, Computer Architecture: A Quantitative Approach,Second Edition, and
have interest in designing reliable benchmarks.

Overview | Types | Engineering | Reporting | Revising | Questions