Benchmarks and Different Aspects of Benchmarking

Overview

Benchmarks are individual programs or a mixture of programs that are run on a target machine to measure the overall performance of the system, or to measure more specific aspects of the performance, such as Graphics Applications, I/O Processing, Net Browsing, etc. Any aspect of computer performance that matters to the user can be benchmarked.

In this project factors affecting the benchmarking and steps involved in benchmark engineering are discussed, and extensive examples of different benchmarks are provided. Techniques to establish criteria for determining the reliability and drawbacks of benchmarks are explained, and discussion on how targeted optimizations and improvements in technology affect the accuracy of benchmarking is also included. Ideas on how to choose appropriate benchmarks to reveal the actual performance behavior of a target machine are also described.

The materials discussed in this project are based on Section 1.5: Measuring and Reporting Performance, and Section 1.8: Fallacies and Pitfalls of the prescribed text book for CMSC 411, Computer Architecture: A Quantitative Approach. The last part of this project is devoted to providing the reader with Questions and Answers, related to the subject discussed.