Introduction to Parallel Computing
326.602, SS 1999, Start: 8.3.1999
Mo 8:30-10:00, T811
The efficient application of parallel and distributed systems
(multi-processors and computer networks) is nowadays an important task for
computer scientists and mathematicians. The goal of this course is to provide
an integrated view of the various facets of software development on such
systems including the most important aspects of
- System architectures,
- Programming languages and models,
- Software development tools,
- Software engineering concepts and design patterns,
- Performance modeling and analysis,
- Experimenting and measuring.
Class presentation will be accompanied by hands-on experience on a
distributed shared memory multiprocessor in
Students are expected to elaborate small programming exercises and to present
them in class; some experience in C programming is assumed.
- data-parallel programming (CONVEX C/Fortran),
- message passing programming (MPI).
A cross-section of the course.
- The Convex SPP System
Architecture and operating system
- Semiautomatic Parallelization
Basic concepts and their application on the SPP.
- Performance of Parallel Programs (PostScript Slides)
Modeling and analyzing the performance of parallel programs, some material
from Chapter 3 of Designing
and Building Parallel Programs
- Parallel Program Design (PostScript
The engineering of parallel programs based on Chapter 2 of Designing
and Building Parallel Programs.
- Distributed Memory Programming (PostScript
Some algorithms for distributed memory multiprocessors.
- The MPI Message Passing Interface (Chapter from Foster's
A tutorial on the standard interface for message passing programming;
see also the user guide on the
MPICH Home Page.
- Parallel Performance Analysis
Measuring parallel program performance, program visualization.
- Miscellaneous Issues
More on parallel architectures, scheduling and mapping, deadlock detection and
avoidance, mutual exclusion: problem and mechanisms, distributed random number
- Exercise 1
Use the automatic parallelization features of the Convex SPP compiler
to solve the "All Pairs Shortest Paths" problem starting with
a sequential solution (
- Exercise 2
- Use the MPI library on the
Convex SPP compiler to solve the "All Pairs Shortest Paths" problem starting
with a sequential solution (
Please show me your preparations one week before the actual presentation!
- May 10
- Thomas Raehalme
"Overview on Current Parallel System Architectures",
Balazs Hadhazi-Borsos "Parallel Computational Geometry"
- May 17
- Hong Gu "Parallelism for Image Understanding",
Sorin Ciufudean "Parallel Computation in
Biomedicine: Genetic and Protein Sequence Analysis"
- May 31
- Hanga Lucian "Parallel Sorting Algorithms", Florina Piori "Data Parallel Algorithms"
- June 7
- Stefan Gerhold "Solving Linear Systems Numerically"
- June 14
- Temur Kutsia "Data Structures for Parallel Processing",
Stefan Katletz "Fast Fourier Transform"
- June 21
- Igor Rents "Parallel Graph Algorithms"
- June 28
- Jürgen Hartl "Parallel Genetic Algorithms"
Cleopatra Pau "Combinatorial Search"
- Ian T. Foster
- Designing and Building Parallel Programs --
Concepts and Tools for Parallel Software
Engineering, Addison Wesley, Reading, MA, 1995.
- Michael J. Quinn
- Parallel Computing -- Theory and Practice,
2nd edition, McGraw-Hill, New York, NY, 1994.
- Kai Hwang
- Advanced Computer Architecture: Parallelism,
Scalability, Programmability, McGraw-Hill, New York, 1993.
- SPP Language and Programming Overview Course
Notes (Condensed Version), CONVEX
Education Center, Richardson, Texas, January 1995.
Maintained by: Wolfgang Schreiner
Last Modification: July 2, 1999