Update Module 2 (from the version used in the 2020 ECP tutorial video recordings) #166

The checklist for suggested revisions contains:

• Slide titles and slide numbers that refer to the Module 2 legacy PDF (2020 version).
  • When making changes, refer to the latest PDF built from CI.
• People in parenthesis who weighed in on the discussion during the Education and Training working group meetings and are potential contributors for the suggested change. Items are pulled from the following meetings of the working group:
  2026-06-16, 2026-06-23, 2026-06-30.

• In all slides:
  • Get rid of dates in footer. Okay to have PDF version tags. Maybe keep month and year. (Patrick, Christian)
• [Hari working on this:] In "Module 1: Summary" (slide 4/60):
  • Get rid of the summary. Any instructor normally repeats what they did in a lecture. Also obstacle to mixing modules (Christian, Pariksheet) Remove previous module summaries at the beginning of modules 2 and 3 #207
  • Not sure if this should also apply to the preceding slide on lecture series outline? Should the outline be auto-generated? (Pariksheet)

Views

• In "View motivation" (slide 8/60): [Hari: done in PR Replace DAXPY by vector addition #191]
  • Replace DAXPY with "vector add", because not sure if [LAPACK] DAXPY is universally known. (Christian)
  • Currently,we assume the _x a nd _y vectors allocated on the CPU fail when trying to access them from GPU. However, a Complication these days is with HMM [Heterogeneous Memory Management] where it does not actually fail but is slow. Need a comment somewhere of it not being true anymore. (Christian)
  • Christian glosses over certain details that would be confusing to teach at this stage; add LaTeX Beamer \note{...} to guide instructors in an instructor-specific PDF build of the slides about sign-posting anything that will be covered in a future module / exercise. The notes can be enable with an option like \setbeameroption{show notes on second screen=right} (Pariksheet, after SC26)
• [Hari working on this:] In "Views (2)" (slide 11/60):
  • Use "reference semantics" in place of shallow / deep copy that may not be widely known. Could say "Views have reference semantics and assigning them does not copy the data they point to". (Pariksheet, Christian) About shared_ptr like behaviour of Views #206
  • Add a illustration showing the copy. Introduce a new slide which walks through the example with a graphic. Currently doing that verbally, but could go through step-by-step. (Rahul, Christian) Illustrate copying of Views #205
• In "Views (3)" (slide 12/60): [Hari: done in Update slide on View properties of Module 2 #200]
  • Remove assert(a.data() != nullptr); because people shouldn't be doing that anymore. It's part of interoperability with raw arrays using unmanaged views. (Christian)
  • Add a link to the documentation of Kokkos Views. (Christian)
  • Add a "Rank" function: static_assert(a.rank() == 2); (Christian)
  • Add the is_allocated() function (which generally does a.data() != nullptr). (Christian, Rahul)
  • Consider instructor LaTeX Beamer \note{...} for functions like data_handle and mapping that returns layout. And accessor function. But we haven't introduced these concepts yet and they are much less used than this. (Christian)
• In "Exercise # 2: Inner Product, Flat Parallelism on the CPU, with Views" (slide 13/60):
  • Consider removing any reference to memory spaces (i.e. "GPU with UVM" / "GPU with shared space" / Kokkos::SharedSpace in exercise_2_begin.cpp file) that we have not talked about yet because later we want learners to plug in SharedSpace memory explicitly to run the example on the GPU. (Christian)
  • If we decide to keep shared memory space, allow advanced learners to compile this exercise with GPUs by adding Kokkos::SharedSpace and compile with -D Kokkos_ENABLE_CUDA=ON or -D Kokkos_ENABLE_${GPU}=ON.
  • If we decide to keep shared memory space, need another way of having separate executables / editing exercise_2_begin.cpp file for separate 02_Exercise.host and 02_Exercise.cuda (rename to 02_Exercise.gpu?) targets; these targets were from using Makefiles before the current CMakeLists.txt file. Christian has an idea of what he wants to do about this. When Pariksheet tried, it the CMakeLists.txt seems to pick up whatever the spack Kokkos package was compiled with. (Pariksheet, Christian)
• In "Advanced features we haven’t covered" (slide 14/60): [Christian PR View last slide: Coming Next slide update #187]
  • Change title to say we are going to cover it. (Christian)
  • Change "Memory traits to control access modalities; covered later". (Rahul, Pariksheet, Christian)

Execution and Memory Spaces

• In "Execution spaces (1)" (slide 16/60):
  • Add instructor LaTeX \note{...} that execution space is not just a representation of physical resources but a mechanism to leverage them; for example, serial and OpenMP run on the same cores with two different mechanisms. (Pariksheet, Christian, after SC26)
  • Remove "NUMA Domain" because the user does not need to know about it right now. They will need to know about it in the future and the same picture is used later with different colors. (Rahul, Christian)
• In "Execution spaces (2)" (slide 17/60):
  • Talk about host execution space later and for now limit this slide to covering host process. We have the type DefaultExecutionSpace which we should introduce earlier. Revise wording of "host", "host process" and "default [host] execution space" to not oversimplify: for example, the red lines highlighted in the code may be executed in a Serial instance's own std::thread which is not the same as the host process thread and also OpenMP may not include the host process thread (Christian, Rahul)
• In "Execution spaces (3)" (slide 18/60):
  • Rename numberOfIntervals to n so that we have enough space to also explicitly specify RangePolicy with DefaultExecutionSpace type. Explicitly say that RangePolicy<>(0, n) runs on DefaultExecutionSpace. (Rahul, Christian)
• In "Memory spaces (0)"(slide 21/60):
  • Think about how to make clearer that memory space includes more than host and device memory. Could put in parentheses: hardware, software, movement semantics. Could say it's equivalent to allocators in C++. (Christian, Pariksheet)
• In "Memory spaces (1)" (slide 22/60):
  • Rework DefaultMemorySpace and DefaultHostExecutionSpace in this and previous slides. Need to introduce DefaultMemorySpace and DefaultHostMemorySpace to explain the concept "memory space [is] set at compile time". HostSpace always exists and is guaranteed, but [in the code at the bottom of the slide] we only have DefaultExecutionSpace one time and also not sure it's correct C++ with maybe a missing type name.
  • Don't want people to use CudaSpace, CudaUVMSpace at all beacuse it's only intended for interoperating with raw Cuda code from Kokkos. At the least say they are non-portable. (Christian)
• In "Memory spaces (2)" (slide 23/60):
  • Replace CudaSpace with generic GPUSpace hopefully by November; have to introduce an alias for DefaultExecutionSpace::MemorySpace. (Christian, Patrick)
• In "Execution and Memory spaces (0)" (slide 24/60): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Rename "Anatomy of a kernel launch" to "Steps of a kernel launch". (Patrick, Christian)
  • Make clear that metadata is part of the functor being copied. (Christian)
• In "Execution and Memory spaces (2)" (slide 26/30): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Use the available space to put more explicitly on the slide that while we cannot access the data that points back to the host we can query the metadata without dereferencing it. For example, can ask for host.extent(0) and say "Metadata is getting copied and accessible, but deferenceability depends on where the data lives".
• In "Execution and Memory spaces (3)" (27/60): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Make more explicit what is going on so they don't have to watch the video. Change "fault" to a C++ comment of "initialization on the host" to explain what is happening. (Patrick, Christian)
• In "Execution and Memory spaces (4)" (28/60): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Fix "illegal access" to use comment syntax to make it valid C++ code. (Patrick, Christian)
  • In the 3 approaches at the bottom of the slide, use more portable versions instead of "Cuda". (Christian)
• In "Execution and Memory spaces (5)" (29/60): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Change "Cuda" to "Cuda/HIP/SYCL" and "runtime" even to "hardware" [that] handles data movement. (Christian)
  • Publish measurements of when to use shared memory from Christian's HPSF talk and then add a link here to point to those details. (Patrick, Luc, Christian)
  • Add "General rule of thumb is you only initialize it once and touch once then fine". (Christian)
• In "Mirrors of Views in HostSpace" (slide 32/60): [Luc: done in PR Module 02: various updates to Execution and Memory spaces section #190]
  • Consider adding "On APUs like MI300A this mode will also avoid multiple allocations". (Christian)

Managing memory access patterns for performance portability

• In "Example: inner product (0)" (slide 36/60): [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Fix code getting clipped by the illustration. (Christian)
• In "Example: inner product (1)" (slide 37/60): [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Add comment at bottom for advanced people of "Layout left padded dynamic extent". Could also mention ISO-C++ std::mdspan std::layout_left / std::layout_right. (Patrick, Luc, Christian)
• In "Layout" (slide 38/60) [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Expand the line on "Layouts are extensible: ≈ 50 lines" with "as specified in ISO-C++ using std::mdspan layouts" (Patrick, Christian)
• In "Exercise # 4: Inner Product, Flat Parallelism" (slide 39/60): [Luc: done in PR Exercise 04: using modern names for spaces #185]
  • Generalize use of memory spaces. (Christian)
  • Clean up exercise 4 C++ code; there's a huge commented-out list of execution space and memory space. Primary space is layouts. Secondary goal is using shared space, host space, and things like that. (Christian, Priority for SC)
• In "Exercise # 4: Inner Product, Flat Parallelism" (slide 40/60):
  • Update benchmarked hardware to something more modern. Module 1 has similar graphs. (Christian, Pariksheet)
• In "Caching and coalescing (1)" (slide 42/60):
  • Mark on slide "own" and "shared" cache. CPU thread 0 wants all access in cache line, GPU wants to collaboratively access cache because it's all one cache. (Patrick, Pariksheet, Christian)
• In "Caching and coalescing (2)" (slide 43/60): [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Instead of CudaSpace have GPUMemorySpaces? (Christian)
  • (Some disagreement on how worthwhile this is) Could expand caching to say "[...] and hit the same cache line. (Christian, Patrick) But this may make the slide too heavy. (Rahul)
• In "Example: inner product (2–4)" (slides 47–49/60) [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Replace "CudaSpace" with "GPU"; NB: no impolite GPU memory space. (Patrick, Christian)
• In "Memory Access Pattern Summary" (slide 51/60) [Luc: done in PR Module 02: Memory Access Patterns update #188]
  • Replace "CudaSpace" with "GPU". (Christian)

Advanced Reductions

• In "Reducers" (slide 53/60): [Luc: done in PR Module 02: Advanced Reductions updates #189]
  • Make code highlighting persistent. (Patrick)
  • Focus on important code pieces, therefore code like KOKKOS_LAMBDA(...) can be gray. (Christian)
  • Link to documentation here so that people can click on things to go to the wiki or specific or general places in the documentation. Consider other places in the slides to add such links, but doesn't have to be everywhere. (Patrick, Christian)
• In "Simultaneous Reductions" (slide 54/60): [Luc: done in PR Module 02: Advanced Reductions updates #189]
  • Use 3 different colors to highlight the things and lines they belong: Kokkos::Max<float>(max_value) another color for sum on the same line. Highlight the two different sums with different colors, so we have different colors for the sums and the max. Make sure to highlight in the conditional.
• In "Views as Result arguments" (slide 55/60): [Luc: done in PR Module 02: Advanced Reductions updates #189]
  • Highlight h_sum and d_sum to be more clear. (Patrick, Christian)
• Note: the next few slides / slide numbers discussed are not the 2020 legacy PDF but the current PDF: https://kokkos.org/kokkos-tutorials/main/KokkosTutorial_02_ViewsAndSpaces.pdf
• In "Custom Reductions"–"Reducer Concept" (slides 56–58/64): [Luc: done in PR Module 02: Advanced Reductions updates #189]
  • Instead of having these slides, point to "How to write a custom reducer" in the documentation or for "full requirements for writing a customer reducer" look here.
  • Could use different colors for init, detail, and final block. (Patrick)
  • Detail block is specific to GPU but not necessarily CPU. Could instead say "Reduce all thread-local variables and produce one total result".

Change log: