Learn software performance engineering
We are actively maintaining the following list of resources to help you learn software performance engineering — developing fast code for applications that run on modern multicore computers. If you have any questions, comments, or suggested additions, please let us know.
Introductory materials
- Introduction to Parallel Computing: This overview briefly introduces a broad range of topics and concludes with its own list of recommended reading. (By Livermore Computing)
- Parallel Computing for Beginners: This free on-line textbook is designed to teach early computing students the basics of parallel and distributed computing. (By Joel C. Adams, Richard A. Brown, Suzanne J. Matthews, and Elizabeth Shoop of CSinParallel)
Tutorials and interactive books
- Programming Parallel Computers: Open online version of the Aalto University course CS-E4580, which teaches how to write fast code for modern computers. (By Jukka Suomela)
- Performance Ninja Class: This 27-module curriculum focuses on low-level performance issues and requires basic C++ skills. (By Denis Bakhvalov)
- Intermediate Parallel and Distributed Computing: This sequel to Parallel Computing for Beginners introduces numerous patterns of parallel and distributed computation. (By Elizabeth Shoop)
- NERSC training materials: If you're looking for something specific (as opposed to following a curriculum), check out the archives of past training events hosted here. (For example, here is a 5-part series from 2023 about programming GPUs with the HIP parallel computing platform.)
- Parallel computing e-book: Johns Hopkins course EN 601.420/620, "Parallel computing for data science," includes a Jupyter-based e-book that you can install on your local machine.
Books
Principles and practice of writing fast code
- Algorithms for Modern Hardware (2023) by Sergey Slotin
- The Art of Multiprocessor Programming (2021) by Maurice Herlihy, Nir Shavit, Victor Luchangco, and Michael Spear
- Performance Analysis and Tuning on Modern CPUs (2020) by Denis Bakhvalov
- Programming Models for Parallel Computing (2015) by Pavan Balaji (ed)
- Performance Solutions (2001) by Connie Smith
Patterns of parallel programming
- Structured Parallel Programming: Patterns for Efficient Computation (2012) by Michael McCool, James Reinders, and Arch Robison
- Patterns of Parallel Programming with .NET (2010) by Stephen Toub
- Patterns for Parallel Programming (2004) by Timothy Mattson, Beverly Sanders, and Berna Massingill
- Our Pattern Language by UC Berkeley