Parallel Programming with OpenACC

Description

Parallel Programming with OpenACC is a modern, practical guide to implementing dependable computing systems. The book explains how anyone can use OpenACC to quickly ramp-up application performance using high-level code directives called pragmas. The OpenACC directive-based programming model is designed to provide a simple, yet powerful, approach to accelerators without significant programming effort.

Author Rob Farber, working with a team of expert contributors, demonstrates how to turn existing applications into portable GPU accelerated programs that demonstrate immediate speedups. The book also helps users get the most from the latest NVIDIA and AMD GPU plus multicore CPU architectures (and soon for Intel® Xeon Phi™ as well). Downloadable example codes provide hands-on OpenACC experience for common problems in scientific, commercial, big-data, and real-time systems.

Topics include writing reusable code, asynchronous capabilities, using libraries, multicore clusters, and much more. Each chapter explains how a specific aspect of OpenACC technology fits, how it works, and the pitfalls to avoid. Throughout, the book demonstrates how the use of simple working examples that can be adapted to solve application needs.

Presents the simplest way to leverage GPUs to achieve application speedups
Shows how OpenACC works, including working examples that can be adapted for application needs
Allows readers to download source code and slides from the book's companion web page

Front Cover 2
Parallel Programming with OpenACC 5
Copyright 6
Contents 7
Contributors 13
Foreword by Michael Wolfe 25
Preface 29
Acknowledgments 31
Chapter 1 From serial to parallel programming using OpenACC 33
- A Simple Data-Parallel Loop 33
  - The OpenACC Kernels Construct Compared to the Parallel Construct 36
  - The Various Forms of OpenACC Parallelism 39
  - Runtime Results for accFill_ex2 40
- A Simple Task-Parallel Example 41
- Amdahl’s Law and Scaling 43

Front Cover 2
Parallel Programming with OpenACC 5
Copyright 6
Contents 7
Contributors 13
Foreword by Michael Wolfe 25
Preface 29
Acknowledgments 31
Chapter 1 From serial to parallel programming using OpenACC 33
- A Simple Data-Parallel Loop 33
  - The OpenACC Kernels Construct Compared to the Parallel Construct 36
  - The Various Forms of OpenACC Parallelism 39
  - Runtime Results for accFill_ex2 40
- A Simple Task-Parallel Example 41
- Amdahl’s Law and Scaling 43
  - Big-O Considerations and Data Transfers 45
  - The Scalability of the accTask.cpp code 47
- Parallel Execution and Race Conditions 48
  - Race Conditions 49
- Lock-Free Programming 53
- Controlling Parallel Resources 55
- Make Your Life Simple 57
- References 59
Chapter 2 Profile-guided development with OpenACC 61
- Benchmark Code: Conjugate Gradient 62
  - Building the Code 62
  - Initial Benchmarking 63
- Describe Parallelism 65
  - Accelerating Waxpby 66
  - Accelerating Dot 68
  - Accelerating Matvec 70
- Describe Data Movement 73
  - Describing Data Movement of Matrix and Vector 74
- Optimize Loops 78
  - Reduce Vector Length 80
  - Increase Parallelism 81
- Running in Parallel on Multicore 83
- Summary 84
Chapter 3 Profiling performance of hybrid applications with Score-P and Vampir 87
- Performance Analysis Techniques and Terminology 88
- Evolutionary Performance Improvement 89
- A Particle-in-Cell Simulation of a Laser Driven Electron Beam 90
- Preparing the Measurement Through Code Instrumentation 91
- Recording Performance Information During the Application Run 92
- Looking at a First Parallel PIConGPU Implementation 94
- Optimizing GPU Kernels 96
- Freeing Up the Host Process 96
- Adding GPU Task Parallelism 97
- Investigating OpenACC Run Time Events With Score-P and Vampir 98
- Summary 99
- References 100
Chapter 4 Pipelining data transfers with OpenACC 101
- Introduction to Pipelining 101
- Example Code: Mandelbrot Generator 103
  - Blocking Computation 105
  - Blocking Data Copies 107
  - Asynchronous Execution 108
- Pipelining Across Multiple Devices 109
- Conclusions 114
Chapter 5 Advanced data management 115
- Unstructured Data Regions 116
- Aggregate Types With Dynamic Data Members 118
  - Building Data Structures From the Top-Down 118
- C++ Class Data Management 121
  - Creating a Generic List Container Class 123
  - accList Update Methods 125
- Using Global and Module Variables in Routines 126
- Using Device Only Data 128
  - Mapping Device Data 130
  - Sharing Device Memory Between MPI Processes 131
- Code Examples 134
- Runtime Results 140
- Summary 142
Chapter 6 Tuning OpenACC loop execution 143
- The Loop Construct 144
- Basic Loop Optimization Clauses 145
  - The auto Clause 145
  - The independent Clause 147
  - The seq Clause 148
  - The reduction Clause 149
  - The collapse Clause 150
- Advanced Loop Optimization Clauses 152
  - The gang, worker, and vector Clauses 152
  - The tile Clause 153
- Performance Results 155
- Conclusion 156
Chapter 7 Multidevice programming with OpenACC 157
- Introduction 157
  - Scalability Metrics for Success 158
- Three Ways to Program Multiple Devices With OpenACC 158
  - Single-Threaded Multidevice Programming 159
  - Multithreaded Multidevice Programming 159
  - Handling Device Affinity or Selecting a Device 159
  - Multiprocess Multidevice Programming 160
- Example: Jacobi Solver for the 2D Poisson Equation 161
- Domain Decomposition 162
  - Single-Threaded Multidevice Programming 163
  - Multithreaded Multidevice Programming 167
  - Multiprocess Multidevice Programming 169
    - CUDA-aware MPI 172
  - Overlapping Communication With Computation 172
- Debugging and Profiling 175
  - Debugging With cuda-memcheck and cuda-gdb 175
  - Profiling With nvprof and the NVIDIA® Visual Profiler 176
- Conclusion 176
- References 178
Chapter 8 Using OpenACC for stencil and Feldkamp algorithms 179
- Introduction 179
- Experimental Setup 180
  - Feldkamp-Davis-Kress Algorithm 181
  - 2D Heat Equation 187
- Hybrid OpenMP/OpenACC 194
- Summary 197
- References 198
Chapter 9 Accelerating 3D wave equations using OpenACC 199
- Introduction 199
- Code Example: Solving 3D Scalar Wave Equation 200
- Converting Stack to Heap 202
- Measuring Host Baseline Scalability 204
  - Creating the OpenMP Build Environment 205
  - Running the OpenMP Code 206
  - Compiling the OpenMP Code 206
- Using OpenACC Tools 207
  - Adding OpenACC Parallel Directives 207
  - Compiling With OpenACC Parallel Directives 208
  - Taking Advantage of Unified Memory 209
  - Understanding Unified Memory 209
  - Running With Managed Memory 210
  - Profiling Managed Memory With Visual Profiler 210
- Using OpenACC Data Directives 213
  - Profiling With OpenACC Data Directives 214
  - Compiling Using OpenACC Data Directives 214
  - Running With OpenACC Data Directives 215
- Targeting Multicore Systems With OpenACC 216
  - Compiling for OpenACC Multicore 216
  - Running With OpenACC on Multicore Systems 217
- References 218
- Summary 218
Chapter 10 The detailed development of an OpenACC application 219
- Introducing CloverLeaf 219
  - Hydrodynamics Scheme 219
  - Test Case 220
- Development Platform: Cray XK6 221
- Development of OpenACC CloverLeaf 221
  - Acceleration of Individual Kernels 222
  - Hot Spots 222
  - Acceleration of Multiple Kernels 224
  - Achieving Full Residency on the GPU 225
  - Increasing the Problem Size 227
  - Hybrid MPI/OpenACC 228
  - Version A: Initial Performance 228
  - Comparison of Hybrid MPI/OpenMP 228
  - Version B: Inner Loop Dependencies 230
  - Version C: Nested Loops and Global Variables 232
  - Version D: MultiGPUs, Reducing Hidden Transfers 235
  - Version E: “ACC SYNC WAITS” 236
  - Impact of GPU Optimizations on CPU 238
  - MultiGPU Scalability 240
- Conclusion 241
  - OpenACC on the CPU 241
  - Heterogeneous Computing 243
- Summary 245
- For More Information 246
Chapter 11 GPU-accelerated molecular dynamics clustering analysis with OpenACC 247
- Introduction 247
- Overview of MD Clustering Analysis 248
  - Clustering Algorithms 250
  - Fast Computation of Pairwise RMSDs With QCP 252
  - Atom Selections and Trajectory Data Structures 252
  - Dissimilarity Matrix Calculation 254
- Hardware Architecture Considerations 255
- Implementation 256
  - Performance Evaluation Test Case 256
  - QCP Inner Product Loop 257
  - Memory Layout Optimization 257
  - Hand-Vectorized QCP Inner Product Loop 259
  - Code Adaptation and Use of OpenACC Directives 261
- Performance Results 267
- Summary and Conclusion 269
- Acknowledgments 270
- References 270
Chapter 12 Incrementally accelerating the RI-MP2 correlated method of electronic structure theory using OpenACC compiler di... 273
- Introduction 273
- Theory 276
- Implementation 277
  - Gpu-1 277
  - Gpu-3 279
  - Gpu-2 279
  - Gpu-4 281
  - Gpu-5 281
  - Gpu-6 283
  - Gpu-7 284
  - Gpu-8 286
- Results 286
  - Computational Details 287
    - GPU boost 287
  - Scaling With Problem Size 288
  - Scaling With Number of Devices 289
  - Total Performance 292
- Summary and Conclusion 293
- Acknowledgments 294
- References 295
Chapter 13 Using OpenACC to port large legacy climate and weather modeling code to GPUs 299
- Introduction 299
- Porting Approach: Step by Step 301
  - Simplified Atmospheric Model 302
  - Step 1: Adding Parallel Regions 303
  - Step 2: Managing Data Transfers 305
  - Step 3: Basic Optimizations 309
    - Fine-grain control of parallelism 309
    - Porting the output/reductions 310
    - Removal of automatic arrays 311
  - Parallel Loop Containing Subroutine Calls 312
  - Interfacing With CUDA 314
- Performance Optimization 315
  - Advanced Optimization Techniques 315
    - Loop fusion and reordering 315
    - Scalar replacement 317
    - On-the-fly computation 319
    - Caching 319
  - Results for the Radiation Parameterization 320
  - References 322
- Index 323
- Back Cover 329

Author's affiliation

Rob Farber: Chief Scientist and Co-founder, Black Dog End

Description

TOC

Reviews (0)

Author's affiliation