MCS 706 – High-Performance Computing and Parallel Processing
Categories: Doctorate of Computer Science
About Course
Course Description:
This core course examines the architecture, algorithms, and programming models for high-performance computing. Topics include parallel algorithms, distributed memory systems, and applications in scientific simulations and data-intensive computing. Students will gain hands-on experience with parallel programming techniques and explore the challenges of developing scalable and efficient solutions for high-performance computing environments.
Course Objectives:
- Develop a deep understanding of high-performance computing (HPC) architectures and programming models.
- Learn to design and implement parallel algorithms for distributed and shared memory systems.
- Explore the application of HPC techniques in scientific simulations, data-intensive computing, and other domains.
- Gain practical experience in parallel programming using tools such as MPI and CUDA.
- Prepare for advanced research and development in high-performance and parallel computing.
Course Content
Week 1: Introduction to High-Performance Computing
-
LO1: Define high-performance computing and explain its significance in modern computational science
-
LO2: Describe key concepts such as parallelism, scalability, and performance metrics in HPC systems
-
LO3: Analyze the role of HPC in solving complex scientific and data-intensive problems
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 2: Parallel Computing Architectures
-
LO1: Explain different parallel computing architectures including shared-memory and distributed-memory systems
-
LO2: Describe architectural design principles influencing performance in HPC environments
-
LO3: Analyze trade-offs between different parallel architectures for specific computational tasks
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 3: Parallel Programming Models
-
LO1: Define major parallel programming models such as data parallelism and task parallelism
-
LO2: Describe programming paradigms used in HPC including MPI and OpenMP
-
LO3: Analyze the suitability of programming models for different types of parallel workloads
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 4: Parallel Algorithms and Performance Analysis
-
LO1: Explain the design principles of parallel algorithms and their computational complexity
-
LO2: Describe performance evaluation metrics such as speedup, efficiency, and scalability
-
LO3: Analyze performance bottlenecks in parallel algorithm implementation
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 5: Message Passing Interface (MPI)
-
LO1: Explain the fundamentals of MPI communication mechanisms
-
LO2: Describe point-to-point and collective communication operations in MPI
-
LO3: Analyze real-world applications of MPI in distributed computing environments
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 6: Shared Memory Programming with OpenMP
-
LO1: Explain the OpenMP programming model and its directive-based parallelization approach
-
LO2: Describe synchronization mechanisms such as barriers and critical sections
-
LO3: Analyze performance optimization techniques in shared-memory parallel programs
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 7: GPU Computing and Accelerators
-
LO1: Explain the architecture and programming model of GPUs in HPC
-
LO2: Describe CUDA and other accelerator-based programming frameworks
-
LO3: Analyze the performance advantages of heterogeneous computing systems
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 8: Midterm Test
-
Midterm Test
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
Week 9: Distributed Computing Systems
-
LO1: Explain the structure and functionality of distributed computing clusters
-
LO2: Describe scheduling, resource allocation, and load balancing techniques
-
LO3: Analyze challenges in designing scalable distributed HPC systems
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 10: HPC Storage Systems and Data Management
-
LO1: Explain parallel file systems and data storage architectures in HPC environments
-
LO2: Describe strategies for managing large-scale scientific datasets
-
LO3: Analyze performance implications of I/O bottlenecks in HPC workflows
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 11: Fault Tolerance and Reliability in HPC
-
LO1: Explain fault tolerance mechanisms in large-scale parallel systems
-
LO2: Describe checkpointing and recovery strategies used in HPC environments
-
LO3: Analyze system reliability challenges in exascale computing infrastructures
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 12: HPC Applications in Scientific Computing
-
LO1: Describe applications of HPC in simulations, modeling, and data analysis
-
LO2: Explain computational techniques used in scientific research domains
-
LO3: Analyze case studies demonstrating HPC impact on scientific discovery
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 13: Energy Efficiency in High-Performance Computing
-
LO1: Explain energy consumption challenges in large-scale computing systems
-
LO2: Describe strategies for optimizing energy efficiency in HPC architectures
-
LO3: Analyze trade-offs between performance optimization and energy sustainability
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 14: Emerging Trends in HPC
-
LO1: Summarize emerging technologies such as quantum computing and exascale systems
-
LO2: Describe evolving architectures and software frameworks in HPC research
-
LO3: Evaluate future challenges and opportunities in high-performance computing
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 15: Course Review
-
LO1: Summarize core concepts of parallel architectures, programming models, and performance optimization
-
LO2: Analyze interdependencies among HPC components and system design strategies
-
LO3: Evaluate readiness for advanced HPC research and implementation
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
Week 16: Final Test
-
Multiple-Choice Questions
-
True/False Questions
-
Scenario-Based Multiple-Choice Questions
-
Key Terms & Concepts Questions
-
Short Answer Questions
-
Written Assignment
-
Presentation Task
-
Role-Playing Activity
-
Peer Review Task
-
Exercises and Activities Adaptation
