This lesson has passed peer-review! See the publicationnone.

Understanding Package Performance

As parallel packages for computational science become more sophisticated, it becomes more difficult for a researcher to understand the most important factors that determine end-to-end productivity from initial input data to final result. Aspects such as file IO and data transfer can be just as important in practice as the performance and parallel scalability of the application itself. This course will provide an introduction to understanding your research workflow, the place of HPC application performance within the workflow, an introduction to benchmarking parallel applications and how you can use benchmark data to make decisions on running your research on HPC systems.

The lesson aims to answer the following questions:

General Information

Where: This course will be taught online via Blackboard Collaborate. All attendees will be sent the joining link prior to the event.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below). They are also required to abide by the ARCHER2 Training Code of Conduct.

Accessibility: We are committed to making this workshop accessible to everybody.

Materials will be provided in advance of the workshop. If we can help making learning easier for you please get in touch (using contact details below) and we will attempt to provide them.

Contact: Please email support@archer2.ac.uk for more information.


Prerequisites

You should have used remote HPC facilities before. In particular, you should be happy with connecting using SSH, know what a batch scheduling system is and be familiar with using the Linux command line. You should also be happy editing plain text files in a remote terminal (or, alternatively, editing them on your local system and copying them to the remote HPC system using scp).

Requirements

Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed in the Setup section below). They are also required to abide by the ARCHER2 Training Code of Conduct.

Note that this lesson uses The Carpentries template but this is not a Carpentries lesson.

Schedule

Setup Download files required for the lesson
09:00 1. Welcome What can I expect from this course?
How will the course work and how will I get help?
How can I give feedback to improve the course?
09:20 2. Understanding your HPC workflow What are the components of my HPC research workflow?
What impact would performance improvements have on the different components?
How can I understand the dependencies between the different components?
10:20 3. Overview of ARCHER2 and connecting via SSH What hardware and software is available on ARCHER2?
How does the hardware fit together?
How do I connect to ARCHER2?
10:40 4. Break Break
10:55 5. HPC performance and benchmarking Why should I benchmark my use of HPC?
What are the key benchmarking concepts that I should understand?
What is the right performance metric for my HPC use?
What parameters can affect the performance of my applications?
11:50 6. Analysing performance How do I analyse the results of my benchmarking?
Which metrics can I use to make decisions about the best way to run my calculations?
What are the aspects I should consider when making decisions about parallel performance?
What stops my HPC use from scaling to higher node counts?
12:50 7. Lunch Break
13:50 8. Automating performance data collection and analysis Why is automation useful?
What considerations should I have when automating collecting performance data?
Are there any tools that can help with automation?
15:00 9. Break Break
15:15 10. Introduction to profiling How does profiling differ from performance benchmarking?
WHat information can I get from profiling?
How can I profile the application I am using?
16:05 11. Where next? What are the next steps for me?
16:15 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.