Review form for CSEM-reviews 2003 ================================= Paper 7, "Compiler Optimization-Space Exploration" by S. Triantafyllis, M. Vachharajani, B. Vacharajani D.I. August Reviewer: Peter Aronsson Summary ======= The paper present a novel approach of controlling the optimizations of a compiler to increase code efficiency. The approach uses profiling, followed by iterative compilation using different optimization configurations interleaved with a performance estimation of the resulting code. The number of optimization configurations are very high (when considering an exhaustive search) and therefore a pruning technique based on training of a initial set of configurations on a training set is performed. These configurations are then used to build a tree of configurations where individual units of code (functions) are also considered. The approach called OSE is evaluated on two set of benchmarks and give speedup figures for both execution time and compilation time compared to standard compilation. A comparison against exhaustive search of the best optimization configuration is also presented, that illustrate the effect of the pruning of the search space. Finally, some analysis of two case studies indicate that the proposed approach will increase code quality where earlier approaches failed to do so. Main contributions ================= The main contribution is a new method for pruning the state space search for compiler optimization configurations. This is combined with incremental compilation, profiling and individual configurations for each code unit. This results in more efficient code being generated with shorter compilation times compared an exhaustive search. The approach also handles the weaknesses of predictive heuristics for choosing compiler optimizations by a having a performance estimator and using iterative compilation. The paper also contributes with the performance estimator. Merits and Weaknesses ===================== The suggested approach is well evaluated and present good results. The result is a combination of earlier approaches: iterative compilation, profiling, and a presented performance estimator. The paper shows that the pruning of the search space can be done efficiently and still find good optimization configurations. A weakness in the paper is the performance estimator, which is not so well grounded in related work and is in the paper identified as a target for improvements. No comparison with other performance estimation algorithms is given. Numerical ratings ================= Significance 7 Originality 7 Interest to Journal on programming lanugage and compiler technology 6 Quality of experimental evaluation 8 Overall organization 7 Presentation 8 Length 5 References appropriate 7 Overall rating 7 Recommendation : weak accept Reviewers confidence: 6 Suggestions for improvement: Present related work on performance estimation to a larger extent. Why not use several performance estimation algorithms from literature to evaluate what precision is needed and how much it will cost in computational complexity.