Table of Content

• Introduction to Parallel Computing
  • Computer Architecture
    • Basic Computer
    • Processors
    • Memory
    • Cache
    • Virtual Memory
    • Inter-leaved Memory
    • Bus and I/O bandwidth
  • High Performance Computing - Parallelism
    • Granularity
  • Traditional Computer Classification
    • Flynn's Taxonomy
  • Fast processors architectures
    • Pipelined Processors
    • Vector Processors
    • Super-Scalar Processors
  • MIMD - memory organization
    • Shared Memory
    • Distributed Memory

• MPI2 Standard
  • Today's Parallel Systems
  • Parallel Computing
  • Message Passing Standards -- PVM and MPI
  • The MPI2 Software Model
  • Multi-purpose Daemon (MPD)
  • Communication Ring
  • Running Jobs Under MPI2
  • Subtleties in Establishing the Communication Ring Daemons
  • References
  • Examples

• Elementary MPI
  • What is MPI?
  • Binding to C and Fortran
  • MPI Initialization
  • Who Am I?
  • Finalizing MPI
  • Building an MPI code
  • Running an MPI code
  • Examples

• Message Passing
  • Message Content
  • MPI Fortran - C datatypes
  • Communication Protocols

• Point to Point Communication
  • Communication Modes
  • Communication Envelope
  • Deadlock
  • Timing
  • The Cost of Communication
  • Simple examples
  • Example: average of a set of numbers

• Sorting Algorithms
  • Sorting Lists of Data
  • Simplest Algorithm: Bubble Sort
  • Parallel Bubble Sort
  • Quicksort Algorithm
  • Parallel Quicksort Algorithm

• Global Operations
  • Synchronization
  • Broadcast
  • Reduce
  • Sample Code
  • Binary Tree Algori thm - Broadcast

• Monte Carlo Techniques
  • Monte Carlo Integral
  • Calculation of π
  • Applicability of the Method
  • Metropolis Algorithm
  • Guided Monte Carlo Integral
  • Parallel Monte Carlo Algorithm
         Random Numbers Generation        Parallel Monte Carlo Codes
  • Codes

• Programming Strategies
  • Parallel Algorithms, Communications & Load Balancing
  • Static Load Balance
  • Dynamical Load Balance
  • The Mandelbrot Set

• MPE Graphics Environment
  • Introduction
  • C-binding
  • An Aside: Makefile
  • Examples

• Game of Life
  • The Game
  • Rules
  • Sequential
  • Visualization -- Movie
  • Parallel Code

• Numerical Solution of Partial Differential Equations - Introduction
  • Numerical Solution of Partial Differential Equations
  • Finite Difference Approach
  • Laplace and Poisson Equations
  • Numerical Solution
  • ``Exact'' numerical solution
  • Iterative numerical solution
  • About this document ...

• Domain Decomposition
  • Lattice Problems
  • Domain Decomposition
  • Parallel Implementation
  • Elliptic Equation - 1-D Poisson Equation
    • The Model
    • Sequential Implementation
    • Parallel Implementation

• Domain Decomposition - 2D Poisson Equation
  • The Model
  • Serial Implementation
  • Parallel Implementation
  • Red and Black algorithm

• Diffusion Equation
  • Parabolic Equation
  • Model
  • Parabolic PDEs solution
  • Serial Implementation
  • Some Results

• Advanced MPI -- Spawn & RMA
  • Download
  • Spawning Processes
  • Shakespeare
  • Remote Memory Access

• Advanced Communications
  • Classical N-body System
  • User defined MPI structures
  • Packing/Unpacking Data
  • N-body Code Snippets

• Graphics Processing Units (GPUs)
• GPUs
• GPU Based Computing
• borg3.physics.drexel.edu
• CUDA
• Learning CUDA