Schedule, Lectures, and Assignments for EN 600.320/420/620


  • Project 0: AWS Web Server, due 8 February 2018.
  • Project 1: OpenMP Filter, due 22 February 2018.
  • Project 2: Java Multi-threading, due 8 March 2018.
  • Project 3: Game of Life (MPI), due 29 March 2018.
  • Project 4: Three’s Company (Map-Reduce), due 12 17 April 2018.
  • Graduate Project: Investigate a Parallel Something, due 4 May 2018.


(29 January) Introduction and Course Information

The impact of multi-core and pipelines on processing power. Moore’s law reloaded. Overview of modern parallel architectures. Comparing concurrency and parallelism. Obstacles to parallel programming.

(31 January) Introduction and Course Information

Scaling laws and metrics for parallel performance.

(5 February) Factors Against Parllelism and Cloud Introduction

(7 February) OpenMP

An introduction to parallelizing serial programs based on compiler directives. Also serial equivalence, loop parallel constructs, and independent loops.

(12 February) Parallel Architectures

Flynn’s taxonomy. The differences between message passing and shared memory machines. SMPD. Clusters. Hybrid architectures. The parallel memory model and cache hierarchy.

(14 February) Loops

Loop parallel constructs, independent loops, loop dependencies, and loop optimizations.

(19 February) Processes, Threads

Operating system process abstraction: contexts and context switching, virtual memory, program organization in memory. Operating system threads, thread context, thread memory model.

(21 February) Programming with Java Threads

Asynchrony, waiting on threads, volatile variables, and synchronized functions.

(26 February) Synchronization

Mutual exclusion, the Coordinated Attack Problem, Peterson’s algorithm, tournament algorithms, Lamport’s fast mutual exclusion.

(28 February) Safety and Liveness

Bakery algorithm, synchronization primitives, spin locks, ticket algorithm, barriers.

  • Reading:
    • Lecture derived from “Gadi Taubenfeld, Synchronization Algorithms and Concurrent Programming, Prentice Hall, 2006.” Chapters 3,4,5,6.
    • Herlihy, Chapter 2.
  • Slides:

(5 March) Top500 and MPI Introduction

Supercomputer of the world and trends in architecture. MPI Overview, the MPI Runtime Environment.

(7 March) MPI Messaging and Barriers

MPI Synchronous Messaging, Asynchronous I/O, and Barriers.

(12 April) HPC Checkpoints and I/O Performance

(14 April) Introduction to Map/Reduce

The Google Parallel computing environment, functional programming concepts applied to large-scale parallelism, text processing, the Google File System.

(19 and 21 March) Spring Break (stay healthy)

(26 March) Review session and study period

(28 March) Midterm Exam

You are allowed to bring 1 (one) 8.5” x 11” piece of paper with handwritten notes on both sides. Your notes sheet must be handwritten!

(4 April) Hadoop!

(9 April) Spark

Spark and RDD’s.

(11 April) GPU and CUDA Architecture

(16 April) PIG and Hive

(18 April) Class cancelled (Sorry)

(23 April) Roofline

(25 April) CUDA Programming

(30 April) CUDA Optmization: Parallel Reduction