Benchmarks
Performance best determined by running a real application
- Use programs typical of expected workload
- Or, typical of expected class of applications e.g., compilers/editors, scientific applications, graphics, etc.
Small benchmarks
- nice for architects and designers
- easy to standardize
- can be abused
SPEC (System Performance Evaluation Cooperative)
- companies have agreed on a set of real program and inputs
- can still be abused
- valuable indicator of performance (and compiler technology)