TDDC78 literature (2012)

Course material (mandatory):

• C. Kessler:
  Programming of parallel computers - Compendium OHs.
  Revised for 2010.
  Accessible electronically to registered students on this page (login and password sent to registered participants by email).

• L. Elden, H. Park and Y. Saad.
  Scientific Computing on High Performance Computers (compendium),
  version 2008.
  Accessible electronically to registered students on this page (login and password sent by email).
  Reading directions by H. Branden

Complementary reading:

If you prefer to work with a textbook beyond the above material, we recommend the following one, or the alternatives given below:

• B. Wilkinson, M. Allen: Parallel Programming, Second Edition. Prentice Hall, 2005. ISBN 0-13-140563-2.
  Available for loan and as course reference copy in the TekNat library and in the Bokakademin shop on the campus.

Alternative textbooks:

• C. Lin, L. Snyder: Principles of Parallel Programming. Addison-Wesley, 2009.
• Ian Foster: Designing and building parallel programs. Addison-Wesley 1995.
  (the previous course book, out of print, but available online.)
  Errata
  This excellent book still matches the course's level and contents well, but the tools chapter is fairly aged, and newer issues such as OpenMP or Grid Computing are, of course, missing completely.
• T. Rauber, G. Rünger: Parallel Programming: for Multicore and Cluster Systems. Springer, 2010.

These books can be reused in the course TDDD56 Multicore and GPU Programming.

Articles (mandatory):

• W. Hillis, G. Steele: Data parallel algorithms. Comm. ACM 29(12), Dec. 1986
• C. Kessler, J. Keller: Models for parallel computing: Review and Perspectives. PARS-Mitteilungen 24: 13-29, ISSN 0177-0454, Dec. 2007. Gesellschaft für Informatik e.V., Germany.

Online resources:

• The lab kompendium and the lecture notes for the lessons can be found on the TANA77 lab page
• Online version of Ian Foster: Designing and building parallel programs.
• A nice tutorial giving an Introduction to Parallel Computing by B. Barney, LLNL
• TOP-500 list
• Pthreads tutorial at LLNL
• Simple MPI example codes: PI calculation in C pi_mpi.c, in Fortran pi_mpi.f
  Many more MPI source code examples can be found on the web.

• Some open-source MPI implementations (also usable on uniprocessor and multicore systems):
  • MPICH2 for Windows (x86-32, x86-64, IA-64), Unix, also Linux, FreeBSD
  • OpenMPI for Linux (32 and 64bit), OS-X, Solaris
• OpenMP forum (www.openmp.org): contains the official standard documentation, a comprehensive tutorial, a few sample programs, links to OpenMP workshops, benchmarks, and more.

• Some open-source OpenMP compilers:
  • GCC version 4.2 or later
  • OpenUH, Univ. of Houston, USA
  • Open64 Compiler and Tools

• VAMPIR
• Tutorial: Using VAMPIR by Pallas GmbH, 1998
• SCALASCA performance analysis toolset
• A recent survey of tools for parallel debugging and performance analysis can be found in Chapter 4 of:
  High Performance Tools & Technologies by Michael Collette, Bob Corey, and John Johnson, Lawrence Livermore National Laboratory, Technical report UCRL-TR-209289, December, 2004.

• Open Grid Forum

• Slides of Lecture 14: Grid computing by Leif Nixon, NSC

Further reading:

• Rohit Chandra, Leonardo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon:
  Parallel Programming in OpenMP.
  Morgan Kaufmann Publishers, 2001.
  The book is available in the TekNat library.

• Barbara Chapman et al.: Using OpenMP - Portable Shared Memory Parallel Programming. MIT press, 2007.
  Covers OpenMP 2.5

• Jörg Keller, Christoph Kessler, Jesper Träff:
  Practical PRAM Programming.
  Wiley Interscience, 2001.
  Chapter 2 gives a summary of the PRAM model and introduces basic theory of parallel processing (Time, work, cost, Amdahl's law, Brent's theorem, isoefficiency). The book is available in the TekNat library.

• W. Petersen and P. Arbenz:
  Introduction to Parallel Computing. A Practical Guide with Examples in C.
  Oxford University Press, 2004.
  Covers exclusively numerical algorithms. Examples in MPI and OpenMP. A text for those who want to specialize in computational science and numerical analysis. Probably most interesting for TANA77 students. The book is available in the TekNat library.

• Thomas L. Casavant and Jon G. Kuhl:
  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems.
  IEEE Trans. on Software Engineering 14(2), Feb. 1988.
  A survey as presented in the lecture on scheduling.

• J. Hennessy, D. Patterson: Computer Architecture, A Quantitative Approach, 4th ed.. Morgan Kaufmann, 2007. The second edition (1996), third edition (2002) and fourth edition (2007) each contain a chapter on multiprocessor architecture. The library (kvartersbibl. B) has copies. As an alternative:

• D. Culler, J. Singh, A. Gupta: Parallel Computer Architecture. Morgan Kaufmann, 1998. Contains an in-depth treatment of SMP and CC-NUMA shared-memory architectures.

• Articles on concepts and trends in high-performance computing:
  
  • IEEE Computer 30(9), Sep. 1997 Special issue on Chip multiprocessors, and IEEE Micro Sep/Oct 1997 on simultaneous multithreading.
  • On Beowulf clusters:
    T. Sterling: The scientific workstation of the future may be a pile of PCs, Communications of the ACM 39(9), Sep. 1996.
  • IEEE Computer 40(3), March 2007 Special issue on High-Performance Reconfigurable Computing with articles on FPGA accelerators and GPU programming.
  • B. Ghahramani, M.A. Pauley: Java in high performance environments (short overview). IEEE Computer 36(9):109-111, Sep. 2003.
  • J. Gebis, D. Patterson: Embracing and Extending 20th Century Instruction Set Architectures. IEEE Computer 40(4): 68-75, April 2007. Summary of SIMD instruction set extensions in standard microprocessors.
  • M. Hill, M. Marty: Amdahl's Law in the Multicore Era. IEEE Computer 41(7): 33-37, July 2008.
  • A. Grimshaw et al.: An Open Grid Services Architecture Primer. IEEE Computer 42(2): 27-34, Feb. 2009.

Page responsible: Webmaster
Last updated: 2012-03-06