Tutorial B Information

Title: Performance Evaluation in Peer-to-Peer Networks by Marcel Waldvogel and Paul Hurley (IBM Research Zurich , Switzerland)

Abstract:
A lot of exciting research emanates from the peer-to-peer research community. But, when evaluating the performance, it is almost as if we are overcome by the ad-hoc nature of the underlying (or overlaying) networks themselves. We rush and construct experiments, forgetting to define sensible goals ahead of time. Consequently, the generated results may be irreproducible and conclusions unqualified.
The tutorial will focus on the steps required for the scientific method in measurement and simulation analysis of peer-to-peer networks. The basic statistical techniques particularly applicable in this field will be presented. The stages of experiment design and refinement will be described, and some pitfalls and dangerous lines of thinking illustrated. Some examples of erroneous performance evaluation will also be demonstrated, and the corrective steps shown.
Topics include:

• Experiment design
• Statistical analysis
• Scientific method
• Results interpretation
• Reproducibility
• Scalability and extrapolation
• Pitfalls
• Entertaining examples from real life
• Earnest (plus also amusing) examples from peer-to-peer network analysis