Reducing TPC-H benchmarking time

Abstract

Benchmarking a system can be a time consuming operation. Therefore, many researchers have developed kernels and microbenchmarks. Nevertheless, these programs are not able to capture the details of a full application. One such example are the complex database applications. In this work we present a methodology based on a statistical method, Principal Component Analysis, in order to reduce the execution time of TPC-H, a decision support benchmark. This technique selects a subset of queries from the original set that are relevant and may be used to evaluate the systems. We use the subsets to determine the ranking of different computer systems. Our experiments show that with a small subset of 5 queries we are able to rank different systems with more than 80% accuracy in comparison with the original order and this result is achieved with as little as 20% of the original benchmark execution time. © Springer-Verlag Berlin Heidelberg 2005.