Course Syllabus

Course Description

CS 5220 is an introduction to performance tuning and parallelization, particularly for scientific codes. Topics include:

  • Single-processor architecture, caches, and serial performance tuning
  • Basics of parallel machine organization
  • Distributed memory programming with MPI
  • Shared memory programming with OpenMP
  • GPU programming with CUDA
  • Parallel patterns: data partitioning, synchronization, and load balancing
  • Examples of parallel numerical algorithms
  • Examples of parallel algorithms in science and engineering

Students should be able to read and write serial programs written in C++ or related language and be familiar with computer architecture (e.g., CS3410 at Cornell). Because our examples will be drawn primarily from engineering and scientific computations, some prior exposure to numerical methods is useful, though not necessary.

Prior exposure to parallel programming is not required, and non-CS students from fields involving simulation and data-intensive applications are particularly welcome!

Here's the schedule for the lectures and homework: CS 5220 SP24

This lecture schedule should be considered tentative and subject to change, at least until it takes place! The changes will be reflected directly in the schedule spreadsheet.

Logistics

CS 5220, Spring 2024:  Applied High-Performance and Parallel Computing
Classes: TuTh 1:25-2:40 pm ET, Thurston Hall 203

Prof: Giulia Guidi
E-mail: gguidi@cornell.edu
OH: Thursday 3-4 pm in Gates Hall 437 or by appointment.

TA: Valeria Marques Valdivia
E-mail: vmm49@cornell.edu

OH: Tuesday 12-1pm in Rhodes Hall 404.

TA: James Chen
E-mail: bc526@cornell.edu
OH: Monday 9-10 am in Rhodes Hall 406

Course Discussion System

This term we will be using EdDiscussion for class discussion. Rather than emailing questions, I encourage you to post your questions on EdDiscussion.

Coursework

Lecture slides will be posted online after class. Lectures will not be recorded. 

The coursework consists of:

  • Lectures: In-person in Thurston Hall 203 TuTh 1.25-2.40 pm ET.
  • Homework Assignments: HW0 is non-coding homework. HW1-5 are more significant homework, they require students to parallelize code, tune, and perform performance experiments and analysis with different programming models. Instructors expect each programming assignment HW to require more than a day of work. So make sure you do not start working the day before the deadline or you will not have enough time. You have 2 weeks for each coding HW, so use this time wisely. HW0-5 (i.e. all of them) will be worked on individually by each student (see Collaboration below). HW 1-5 will have intermediate graded deadlines to encourage you to start working on the homework early. 
  • Project: in addition to homework, students will complete a significant final project of their choosing. We will suggest final projects, or students may bring in their own projects. Group projects (2-3 people) are highly recommended. If you would like to do the final project on your own, please contact the staff and explain why. Staff will determine if this is an option or not.

Grading

Graded work will be weighted as follows:

  • Class participation: 10%
  • Homework: 55%
    • HW0: 5%
    • HW1: 10%
    • HW2: 10%
    • HW3: 10%
    • HW4: 10%
    • HW5: 10%
  • Final project: 35%
    • Proposal: 5%
    • Project: 20%
    • Presentation: 10%

Your participation makes the class more interesting and useful for all of us. There are several ways to participate:

  • Coming to class;
  • Engaging in group activities in class;
  • Actively engaging and answering questions on EdDiscussion;
  • Helping correct and clarify class notes and assignments.

The lowest grade in HW1-5 will be dropped. Only one grade between HW1-5 will be dropped, there are no additions.

Late Policy

1.5% of the assignment's worth is deducted every day past your due date. NO CREDIT after 7 days from the dye date.

The late policy applies to:

  1. HW0-5

  2. Project Proposal

The late policy does NOT apply to:

  1. Pre-Course Survey

  2. Final Project Poster and Report — in this case, the deadline is STRICT, no late policy, and late submission would result in failing the assignment.

Extension Policy

For each assignment (HW1-5), up to three “slip days” are allowed. Over the semester, you may use a total of six slip days. That is, you cannot use three slip days for every assignment because that would be a total of 15 slip days.

As for the late policy, this policy does not apply to the project.

If you want to use a slip day, write a private post on EdDiscussion (no emails) and the staff will keep track of them.

You will have two weeks to complete each HW1-5 and more than a month for the final project. NO extensions are granted for reasons such as midterms, conference/journal paper deadlines, conference travels, and the like. You should use the slip days when necessary. Emails requesting extensions for these or similar reasons will be ignored.

If you need additional accommodation, ask in writing in advance, with a rationale and a plan for when you will be able to submit the work. I do not guarantee I'll grant you additional accommodation.

Collaboration

An assignment is an academic document, like a journal article. When you turn it in, you are claiming everything in it is your original work, unless you cite a source for it.

You are welcome to discuss homework and projects among yourselves in general terms. However, you should not look at code or write-ups from other students, or allow other students to see your code or writeup, even if the general solution was worked out together. Unless we explicitly allow it on an assignment, we will not credit code or writeups that are shared between students (or teams, in the case of projects).

If you get an idea from a classmate, the TA, a book, another published source, or elsewhere (e.g., ChatGPT), please provide an appropriate citation. This is not only critical to maintaining academic integrity, but it is also an important way for you to give credit to those who have helped you out. When in doubt, cite! Code or writeups with appropriate citations will never be considered a violation of academic integrity in this class (though you will not receive credit for code or writeups that were shared or copied when you should have done them yourself).

Academic Integrity

We expect academic integrity from everyone. School is stressful, and you may feel pressure from your coursework or other factors, but that is no reason for dishonesty! If you feel you can’t complete the work on your own, come talk to the professor, the TA, or your advisor, and we can help you figure out what to do.

For more information, see Cornell’s Code of Academic Integrity.

Accommodations for Students with Disabilities

Students with Disabilities: Your access to this course is important to me. Please request your accommodation letter early in the semester, or as soon as you become registered with SDS, so that we have adequate time to arrange your approved academic accommodations.

Technology/Electronics

Students should not use cellular phones and laptops in lectures unless they receive explicit permission from the instructor (e.g., when using PollEverywhere). The distraction caused by a few students using (or misusing) laptops during lectures far outweighs any benefit. We recognize that there may be circumstances where a particular student may be well served by using a computer in lectures: if this applies to you, please come and talk to me.

AI Policy

You may use ChatGPT for coding only in connection with the Final Project, not for HW0-5. You may use ChatGPT as a tool for developing your project as long as you provide a proper citation and prompt log. If you use ChatGPT, you are responsible for ensuring that the code is correct and consistent with your project proposal and that you understand exactly what the code does and why.

Be aware of the limits of ChatGPT:

  • To achieve high-quality results, you must put in more effort when providing prompts. Refining your prompts is necessary to ensure good outcomes, but it will require some work on your part.
  • Do not trust anything it says. If it gives you a number or a fact, assume it is wrong unless you know the answer or can verify it in other sources. You are responsible for any errors or omissions the tool makes. It works best on topics you understand.
  • AI is a tool, but one you need to acknowledge using. Please include a paragraph explaining what you used AI for and what prompts you used to get results. If you do not, you are in violation of the Academic Integrity Policy.
  • Be thoughtful about when this tool is useful. Do not use it if it is not appropriate for the case or circumstances.

Using ChatGPT or any other AI tool for coding HW0-5 violates the Academic Integrity Policy. 

Perlmutter (aka Course Cluster)

NERSC Perlmutter is the main machine on which homework performance is graded. PSC ACCESS Bridges-2 is meant as a development backup. Do not optimize the code for Bridges-2.

Consider that Perlmutter might be down for 24-48 hours every 2 or 3 weeks, so you mustn't wait until the last minute to start the homework since there will be no extension due to "Perlmutter planned outages". You should check regularly Perlmutter's status and planned outages here.

For unplanned Perlmutter outages, staff will consider an extension depending on (a) when the unplanned outage occurs and (b) how long the unplanned outage lasts.

For example, if an unplanned outage occurs within 3 days of the HW release and lasts less than 48 hours, staff will not grant an extension. On the other hand, if the unplanned outage occurs within 3 days of the due date of HW, staff will grant an extension proportional to the duration of the outage (1-day extension for 1-day outage).

The staff will receive emails from NERSC. So, if there is an unplanned outage, please do not email staff because we already know about it and are already discussing how we will handle it. Please be patient and the staff will notify you of a decision as soon as possible.

Circulating or Selling Class Materials

All materials of this course are copyrighted and it is prohibited to circulate or sell to commercial vendors the course materials, including syllabus, exams, lecture notes, images, presentations, and student papers. Such unauthorized behavior constitutes academic misconduct. Video and/or audio recording of class lectures and review sessions without my permission in advance is prohibited. If you have an accommodation letter from Student Disability Services and are interested in recording for your personal use as a study aid, please make an appointment to meet during OH before you record anything.

Inclusivity Statement

Cornell University (as an institution) and I (as a human being and instructor of this course) are committed to full inclusion in education for all persons. Services and reasonable accommodations are available to persons with temporary and permanent disabilities, to students with DACA or undocumented status, to students facing mental health or other personal challenges, and to students with other kinds of learning challenges. Please feel free to let me know if there are circumstances affecting your ability to participate in class. Some resources that might be of use include:

Emergency Procedures

In the event of a major campus emergency, course requirements, deadlines, and grading percentages are subject to changes that may be necessitated by a revised semester calendar or other circumstances. Any such announcements will be posted to EdDiscussion and Canvas.

Please note that the syllabus will continue to be updated until the beginning of the classes.