Parallel Programming for Modern High Performance Computing Systems

by Pawel Czarnul

In view of the growing presence and popularity of multicore and manycore processors, accelerators, and coprocessors, as well as clusters using such computing devices, the development of efficient parallel applications has become a key challenge to be able to exploit the performance of such systems. This book covers the scope of parallel programming for modern high performance computing systems. It first discusses selected and popular state-of-the-art computing devices and systems available today, These include multicore CPUs, manycore (co)processors, such as Intel Xeon Phi, accelerators, such as GPUs, and clusters, as well as programming models supported on these platforms. It next introduces parallelization through important programming paradigms, such as master-slave, geometric Single Program Multiple Data (SPMD) and divide-and-conquer. The practical and useful elements of the most popular and important APIs for programming parallel HPC systems are discussed, including MPI, OpenMP, Pthreads, CUDA, OpenCL, and OpenACC. It also demonstrates, through selected code listings, how selected APIs can be used to implement important programming paradigms. Furthermore, it shows how the codes can be compiled and executed in a Linux environment. The book also presents hybrid codes that integrate selected APIs for potentially multi-level parallelization and utilization of heterogeneous resources, and it shows how to use modern elements of these APIs. Selected optimization techniques are also included, such as overlapping communication and computations implemented using various APIs.

304 pages, Hardcover

Published February 28, 2018

Reviews

[Piotr Pabis · Rating 4 out of 5]

I would just criticize the technical side of the first release:
* bad layout and ordering: often graphics are far from the text mentioning them and many pages have a single large image with 3 lines of the paragraph below,
* font in algorithms: despite code being written in monospace, in an image with block algorithm, monospace is used for comments while sans-serif is used for code, making unreadable. What's more the subscript and superscript were not rendered properly.