TDDC78 literature (2017)
Course material (mandatory):
Programming of parallel computers - Compendium OHs.
Accessible electronically to registered students on this page (login and password sent to registered participants by email).
L. Elden, H. Park and Y. Saad.
Scientific Computing on High Performance Computers (compendium),
Accessible electronically to registered students on this page (login and password sent by email).
Complementary reading:If you prefer to work with a textbook beyond the above material, we recommend the following one, or the alternatives given below:
- B. Wilkinson, M. Allen: Parallel Programming, Second Edition.
Prentice Hall, 2005. ISBN 0-13-140563-2.
Available for loan and as course reference copy in the Campus Valla library and in the Bokakademin shop on the campus.
- C. Lin, L. Snyder: Principles of Parallel Programming. Addison-Wesley, 2009.
- Ian Foster:
Designing and building parallel programs. Addison-Wesley 1995.
(the previous course book, out of print, but available online.)
This excellent book still matches the course's level and contents well, especially the PCAM method collection for the design of parallel programs, but the chapters on tools and programming models are outdated, topics such as OpenMP, MPI-2 and newer MPI versions, or Grid Computing are missing completely.
- T. Rauber, G. Rünger: Parallel Programming: for Multicore and Cluster Systems. Springer, 2010.
Advanced readingThe following book gives a more in-depth treatment of the topic (beyond what we take up in the course):
- Georg Hager, Gerhard Wellein:
Introduction to High Performance Computing for Scientists and Engineers.
CRC Press, 2010.
Print ISBN: 978-1-4398-1192-4
eBook ISBN: 978-1-4398-1193-1
Campus-Valla library has one printed copy for loan and one e-book copy for electronic loan.
- W. Hillis, G. Steele: Data parallel algorithms. Comm. ACM 29(12), Dec. 1986
- C. Kessler, J. Keller: Models for parallel computing: Review and Perspectives. PARS-Mitteilungen 24: 13-29, ISSN 0177-0454, Dec. 2007. Gesellschaft für Informatik e.V., Germany.
- Online version of Ian Foster: Designing and building parallel programs.
- A nice tutorial giving an Introduction to Parallel Computing by B. Barney, LLNL
- TOP-500 list
- Pthreads tutorial at LLNL
- Simple MPI example codes: PI calculation
in C pi_mpi.c,
in Fortran pi_mpi.f
Many more MPI source code examples can be found on the web.
- Some open-source MPI implementations (also usable on uniprocessor and multicore systems):
- OpenMP forum (www.openmp.org):
contains the official standard documentation,
a comprehensive tutorial, a few sample programs, links to OpenMP workshops,
benchmarks, and more.
- Some open-source OpenMP compilers:
- Tutorial: Using VAMPIR by Pallas GmbH, 1998
- SCALASCA performance analysis toolset
- A recent survey of tools for parallel debugging and performance analysis
can be found in Chapter 4 of:
High Performance Tools & Technologies by Michael Collette, Bob Corey, and John Johnson, Lawrence Livermore National Laboratory, Technical report UCRL-TR-209289, December, 2004.
- Open Grid Forum
- Rohit Chandra, Leonardo Dagum, Dave Kohr, Dror Maydan, Jeff McDonald, Ramesh Menon:
Parallel Programming in OpenMP.
Morgan Kaufmann Publishers, 2001.
The book is available in the TekNat library.
- Barbara Chapman et al.: Using OpenMP -
Portable Shared Memory Parallel Programming.
MIT press, 2007.
Covers OpenMP 2.5
- Jörg Keller, Christoph Kessler, Jesper Träff:
Practical PRAM Programming.
Wiley Interscience, 2001.
Chapter 2 gives a summary of the PRAM model and introduces basic theory of parallel processing (Time, work, cost, Amdahl's law, Brent's theorem, isoefficiency). The book is available in the TekNat library.
- W. Petersen and P. Arbenz:
Introduction to Parallel Computing. A Practical Guide with Examples in C.
Oxford University Press, 2004.
Covers exclusively numerical algorithms. Examples in MPI and OpenMP. A text for those who want to specialize in computational science and numerical analysis. Probably most interesting for TANA77 students. The book is available in the TekNat library.
- Thomas L. Casavant and Jon G. Kuhl:
A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems.
IEEE Trans. on Software Engineering 14(2), Feb. 1988.
A survey as presented in the lecture on scheduling.
- J. Hennessy, D. Patterson: Computer Architecture,
A Quantitative Approach, 4th ed.. Morgan Kaufmann, 2007.
The second edition (1996), third edition (2002) and fourth edition (2007) each contain a chapter on multiprocessor architecture. The (TekNat) library has copies. As an alternative:
- D. Culler, J. Singh, A. Gupta: Parallel Computer Architecture.
Morgan Kaufmann, 1998.
Contains an in-depth treatment of SMP and CC-NUMA shared-memory architectures.
- Articles on concepts and trends in high-performance computing:
- IEEE Computer 30(9), Sep. 1997 Special issue on Chip multiprocessors, and IEEE Micro Sep/Oct 1997 on simultaneous multithreading.
- On Beowulf clusters:
T. Sterling: The scientific workstation of the future may be a pile of PCs, Communications of the ACM 39(9), Sep. 1996.
- IEEE Computer 40(3), March 2007 Special issue on High-Performance Reconfigurable Computing with articles on FPGA accelerators and GPU programming.
- J. Gebis, D. Patterson: Embracing and Extending 20th Century Instruction Set Architectures. IEEE Computer 40(4): 68-75, April 2007. Summary of SIMD instruction set extensions in standard microprocessors.
- M. Hill, M. Marty: Amdahl's Law in the Multicore Era. IEEE Computer 41(7): 33-37, July 2008.
- A. Grimshaw et al.: An Open Grid Services Architecture Primer. IEEE Computer 42(2): 27-34, Feb. 2009.
Page responsible: Webmaster
Last updated: 2017-03-15