Parallel Languages and Programming Models

The general education in programming principles and techniques is essentially based on the sequential model of execution where only one activity is performed at a time and a program describes the succession of activites that lead to the desired results.

However with the emerge of parallel computers (multi-processors) and distributed systems (computer networks) it has become important to coordinate multiple activities that are performed simultaneously and mostly independent of each other.

The goal of this course is to provide an integrated overview over the variety of programming models and paradigms that have been developed and suggested for this purpose. It is not a course on parallel programming with concrete languages but emphasizes the thinking models underlying the variety of languages.

Related Course

Contents

Introduction

• Parallel architectures.
• Distributed systems.

Lecture Notes (ASCII)

Data-Parallel Models

"The world consists of arrays and matrices to be processed."

• SPMD, SIMD.
• FORTRAN 90, HPF, C*, NESL.

Lecture Notes (ASCII), Chapter 7 of Foster.

Shared Variable Models

"The world is a flat global store."

• Unix and Mach.
• Processes, locks, barriers, semaphores.
• Threads, POSIX.

Lecture Notes (ASCII), Threads Article

Message Passing Models

"The world is a collection of talking processes."

• OCCAM, PVM, MPI.
• Fortran M, Erlang.
• Unix sockets, RPC, client/server, WWW, Java.

Lecture Notes I (ASCII)
Lecture Notes II (ASCII)
Lecture Notes III (ASCII)

Communication Spaces

"The world contains rooms for communication."

• Monitors, Shared Data Abstractions, CC++
• Linda and friends.

Lecture Notes I (ASCII)
Lecture Notes II (ASCII)

Object-Oriented Models

"The world consists of autonomous objects."

• Passive Objects: CC++, COOL, Concurrent Smalltalk
• Active Objects: Actors, POOL, uC++
• Autonomous objects, intelligent messages, messengers, mobile agents.

Lecture Notes I (ASCII)
Lecture Notes II (ASCII)
IBM Aglets Page

Parallel Functional Models

"The world consists of expressions to be evaluated."

• Id, Sisal, pH, parallel Haskell, pD.
• Dataflow, graph reduction models.
• Skeletons.

Lecture Notes (ASCII)
Para-Functional Programming Paper
Functional Programming Archive

Parallel Logic Models

"The world consists of propositions to be proved."

• Parallel Prologs.
• GHC Languages, Concurrent Prolog, PARLOG, Strand.
• Constraint Logic Programming (CLP).

Lecture Notes (ASCII)