Schedule, Lectures, and Assignments for EN 600.320/420 01

Assignments

  • Project 0: AWS Web Server, due 14 February 2017.
  • Project 1: Game of Life (MPI), due 2 March 2017.
  • Project 2: OpenMP Filter, due 14 March 2017.
  • Project 3: Measuring Parallel Performance (Java), due 6 April 2017.
  • Project 4: Three’s Company, due 24 April 2017.
  • Graduate Project: Graduate Project, due 5 May 2017.
  • Project 5: (320 only) TensorFlow, due 5 May 2017.

Lectures

(30 January) Introduction and Course Information

The impact of multi-core and pipelines on processing power. Moore’s law reloaded. Overview of modern parallel architectures. Comparing concurrency and parallelism. Obstacles to parallel programming.

(1 February) Concepts in Parallel Processing (1/2)

Measuring parallel performance with scale-up and speed-up.

(8 February) Concepts in Parallel Processing (2/2), MPI

Factors that work against linear scaling: startup costs, interference, skew. MPI Overview, the MPI Runtime Environment.

(13 February) OpenMP

An introduction to parallelizing serial programs based on compiler directives. Also serial equivalence, loop parallel constructs, and independent loops.

(15 February) MPI Continued

MPI Synchronous Messaging, Asynchronous I/O, Barriers, and Collective Operations.

(20 February) Parallel Architectures

Flynn’s taxonomy. The differences between message passing and shared memory machines. SMPD. Clusters. Hybrid architectures. The parallel memory model and cache hierarchy.

(22 February) Loops

Loop parallel constructs, independent loops, loop dependencies, and loop optimizations.

(27 February) Processes, Threads

Operating system process abstraction: contexts and context switching, virtual memory, program organization in memory. Operating system threads, thread context, thread memory model.

(1 March) Programming with Java Threads

Asynchrony, waiting on threads, volatile variables, and synchronized functions.

(8 March) Virtualization

History of virtualization, Traditional VMs, Docker Containers

(13 March) Synchronization

Mutual exclusion, the Coordinated Attack Problem, Peterson’s algorithm, tournament algorithms, Lamport’s fast mutual exclusion.

(15 March) Safety and Liveness

Bakery algorithm, synchronization primitives, spin locks, ticket algorithm, barriers.

(27 March) Introduction to Map/Reduce

The Google Parallel computing environment, functional programming concepts applied to large-scale parallelism, text processing, the Google File System.

(29 March) Map/Reduce, Part 2

Using Hadoop!. The tool-chain. Sorting in Hadoop!. MR as a text streaming environment.

(3 April) Midterm Exam

You are allowed to bring 1 (one) 8.5” x 11” piece of paper with handwritten notes on both sides. Your notes sheet must be handwritten!

(5 April, 10 April) Spark

Spark and RDD’s

(12 April) TOP500, Checkpoints and I/O Performance

(17 April) MR Interfaces: PIG, HIVE

(19 Apr) CUDA Introduction

(24 Apr) CUDA Execution and Memory Model

(26 Apr) CUDA Parallel Reduction

(3 May) NoSQL