Parallel and Distributed Computing Overview and

Syllabus

Professor Johnnie Baker Guest Lecturer: Robert Walker

Instructors• Professor Johnnie W. Baker

– Primary Lecturer

• Professor Robert Walker– Guest lectures on specific architectures– Guest lecture on his groups VLSI work on

parallel architectures

• Possible guest lecturers from parallel processing group– Lecture in areas of expertise– Occasionally cover classes when I am away

Prerequisites

• Designed to be accessible to all graduate students in computer science

• Students who are not CS graduate students may also be qualified to take course

Textbook and References

• Textbook– Parallel Programming in C with MPI and

OpenMP– Michael Quinn, author– Published by McGraw Hill in 2004

• References for Supplementary Reading– Classroom Slides will also include additional

information from a wide range of sources– Any additional needed reference material will

be handed out or posted on course website

Some Features of Course

• Includes coverage of fundamental concepts of parallel computation, rather than focusing only on latest trends.– Often quickly outdated due to rapid

technological changes

• Also covers the currently popular cluster architectures, shared memory processors, and the MP language. – This is the focus of the Quinn Textbook

Course Features (cont.)

• Covers the common techniques for parallel computation by looking at three key features for each technique:– Typical architectural features– Typical programming languages used– Typical algorithm design techniques used.

Some Specific Topics

• Fundamental concepts applicable to all parallel computation.

• Asynchronous (MIMD) distributed memory computation– Message passing communications– Programming using the MPI Language– Architectural features– Examples of typical algorithms

Specific Topics (cont.)• Asynchronous (MIMD) shared memory

computation– Symmetric Multiprocessors or SMPs– OpenMP language overview

• Synchronous Computation – SIMD, vector, pipeline computing– Associative and Multi-Associative Computing– Programming using the ASC language– Programming on WorldScape system with Cn– Fortran 90 and HPF Language overviews– Algorithm examples

Specific Topics (cont.)

• Interconnection Networks– Specific Computer Examples including 2D

mesh, hypercube, etc.– Synchronous and asynchronous

considerations

• Comparing MIMD & SIMD Computation using a Real-Time Application – ATC (Air Traffic Control)

Some Benefits of Course• While principal focus is on parallel computation,

most information is applicable to distributed computing.

• There is a wide choice of thesis and dissertation topics in this area

• Several professors in department work in this area or make major use of parallel computation in their research

• Students working on a thesis or dissertation in another area may benefit from being able to use parallel computation in their work.

Benefits (cont.)• Most large computational problems require

a parallel or distributed system to satisfy the speed and memory requirements

• Parallel computation currently has major advantages over both distributed computation and grid computation for computational intensive problems.– Programs are normally much simpler– Architectures are much cheaper– Grid computing is currently fairly futuristic

Two Complementary Courses

• Parallel & Distributed Computing (Fall)– Architectures– Languages– Parallel Programming– Algorithm Examples for some architectures

• Parallel & Distributed Algorithms (Alternate Springs)– Important Parallel Models– Designing Efficient Algorithms for Various Models– Expect to be next offered in Spring 2009

• PDC and PDA can be taken in either order– Preference is for PDC to be taken first

Real-Time Systems Course

• Will be offered this Spring as an online course

• Co-taught with Professor Drew at Ohio University

• Plan to have a project on implementing a basic ATC system on a WorldScape-ClearSpeed parallel computer.

• The PDC class should provide useful background information for this course.

Assignments and Grading

• Homework assignments– Problems assigned for most chapters– Probably 5-7 different assignments– Some assignments will involve programming

• Course Grade– Based on homework, midterm, and final– Approximate weights

• Homework 40%• Midterm Exam 30%• Final Exam 30%

Documented Disabilities

• If you have documented disabilities, please contact me and make me aware of your needs.

• For information on disability accommodations, support, and verification procedure, please see www.kent.edu/sds

Course Website• Will be established quickly

• Class slides, assignments, and some references will be posted on this website.

• Also, an online reference textbook and a pointer to a second online textbook will be available at this site.

• First Assignment – Read Chapter 1 in textbook.