Talks

Learning MPI means more than learning its syntax. Efficient parallel programs require understanding why communication is necessary but expensive, why an exchange within a node differs from one across the network, and which performance problems arise from communication itself. Such understanding is part of performance analysis, which typically appears late in curricula. Collective operations illustrate this: learners see a single call, while the communication actually carried out by the MPI library remains invisible. This talk presents approaches to teaching MPI in which communication itself becomes the central representation. Based on three years of classroom experience with EduMPI Suite, it discusses the near-real-time visualization of MPI communication, its mapping onto processes, nodes, and cores, and the decomposition of collective operations into their underlying point-to-point communication, which makes the selected algorithm observable. Obtaining this information currently requires a modified MPI implementation, as PMPI intercepts calls at the API boundary and MPI_T, although intended to expose internal information, remains limited in coverage and implementation support. Usability studies and classroom evaluations show which representations change how learners reason about communication, and how visible collective algorithms turn claims about efficiency into observations. In a comparative usability study, 96 % of analysis tasks were solved with EduMPI Suite, compared with 60 % using TAU or CUBE. This experience also indicates what the MPI standard could contribute to make such data more accessible to learning support and performance analysis tools. The same principle extends to asynchronous many-task runtimes, increasingly taught as a higher-level alternative to explicit MPI development. Where MPI serves as their backend, the communication they generate, including collective operations, is again hidden. Making this communication observable is the next step.

Applications invoke collective communication through high-level calls such as MPI_Bcast, MPI_Reduce, or MPI_Alltoall. Internally, MPI libraries commonly implement these operations as algorithms composed of point-to-point communication. The selected algorithm, its data movement, and its suitability for a given process placement and underlying hardware remain hidden. Even established performance-analysis tools offer little support in understanding this behavior: they typically represent a collective operation as a single event, without revealing the internal communication or the algorithm selected by the MPI library. EduMPI Suite makes the internal execution of collective MPI operations visible in near-real-time. Originally developed as an educational tool, it visualizes message transfers between MPI ranks, the placement of the corresponding processes on physical compute nodes, and the associated data volumes. The visualization is updated once per second and presents the evolving communication pattern through 2D and 3D views as well as a communication matrix. The visualization makes algorithmic differences between executions directly observable and shows how message size, process count, process placement, and system topology affect collective communication. This enables the communication behavior of collective communications to be assessed in context: communication hotspots, synchronization effects, waiting states, and inter-node traffic become visible, while efficient combinations of collective algorithm, process placement, and execution configuration can be identified. Beyond conventional MPI applications, the visualization approach could be extended to asynchronous many-task systems (AMT) that use MPI as a communication backend, such as HPX and MPI-based Dagger.jl. In these systems, communication is generated by the runtime rather than explicitly expressed by the developer in the application code. Associating MPI operations with the tasks that initiated them could expose communication patterns hidden beneath the runtime abstraction and reveal recurring fine-grained transfers that could be replaced by suitable collective operations.

Julia is increasingly used in HPC as a single-language alternative to combining high-level scripting with low-level systems languages, but achieving scalable performance still requires expertise in parallel programming. LLMs are increasingly used for code generation and are advancing rapidly with each new version. Yet, existing studies focus on single-shot prompting rather than agentic settings, in which an LLM autonomously plans, generates, and refines code through tool use. Using an OpenCode-based agent extended with a Julia-documentation MCP server, we study agentic generation of parallel Julia code, focusing on task-based execution with Dagger.jl. We evaluate three LLMs, OpenAI GPT-5.5, Anthropic Claude Opus 4.7, and the open-weight Qwen3-Coder-Next, on three problems with distinct parallel structures: approximation, tiled general matrix multiplication, and tiled Cholesky decomposition. The generated Dagger.jl implementations are compared against agent-generated Base.Threads and MPI.jl baselines, with shared-memory experiments scaling to 192 cores and distributed-memory experiments on two nodes. The agents reliably produce executable code for small inputs but fail at larger scales due to deadlocks, oversubscription, or out-of-memory errors, with the open-weight model affected most severely. The two commercial models scale comparably on Base.Threads and MPI.jl, while their Dagger.jl implementations expose recurring weaknesses in task dependencies, granularity, and scheduling. Agentic AI is promising for producing parallel Julia code, but generating robust, performance-aware implementations for large-scale HPC systems remains an open challenge.

High Performance Computing (HPC) enables solving complex scientific and engineering problems that cannot be addressed with conventional systems due to limited computational power or insufficient resources, and is therefore gaining increasing importance in research and industry. Accordingly, HPC has become an integral part of higher education. Efficient use of HPC cluster resources relies on programming models such as the Message Passing Interface (MPI) to develop scalable parallel programs across multiple compute nodes. However, MPI-based development is challenging, as it requires not only solid knowledge of parallel programming, algorithms, and communication mechanisms, but also a deep understanding of performance-critical aspects. Program performance depends on various factors, including hardware and network architectures of HPC clusters. Experienced developers use professional performance analysis tools to identify performance issues. However, using these tools is complex and requires detailed knowledge of cluster architectures, performance metrics, and analysis methods, posing barriers for students. This thesis examines the integration of performance analysis into HPC education and the challenges students face. In this context, EduMPI Suite is introduced as an educational tool for parallel programming with MPI. Through a graphical user interface (EduMPI GUI), learners execute MPI programs on an HPC cluster and receive automated visual feedback on performance-relevant metrics in near-real-time. To support this functionality, EduMPI Suite incorporates a measurement and data-management system comprising an Open MPI fork extended with measurement capabilities and a centralized time-series database. EduMPI Suite is characterized by low measurement overhead (<3.8%), robust and persistent data storage, and low query latency (<0.09 s). EduMPI GUI focuses on visualizing MPI communication and internal algorithmic communication patterns of collective operations. In extensive usability studies and evaluations conducted over multiple semesters, the effectiveness, efficiency, usability, and user satisfaction of EduMPI Suite in identifying performance issues were assessed and compared with established professional tools. The results demonstrate that EduMPI Suite significantly lowers barriers to executing and analyzing MPI programs and is consistently perceived by students as easier and more intuitive than established tools. Students working with EduMPI Suite required substantially less assistance when completing tasks. In a moderated usability study, 96% of tasks related to performance issue identification were successfully completed using EduMPI Suite, compared to only 60% with established tools. EduMPI Suite has proven effective for integrating performance analysis into HPC education.

Performance analysis is essential for understanding the behavior of parallel applications on High-Performance Computing (HPC) systems, identifying bottlenecks and load imbalances. Traditional analysis workflows are post-mortem: applications are instrumented, performance data is collected at runtime, and insights become available only after execution. This workflow is time-consuming - especially for large-scale Message Passing Interface (MPI) applications (i.e., running on many processes/nodes) requiring repeated analysis after code changes - and demands substantial expertise, posing a steep learning barrier for beginners. To address these challenges, in previous work, we proposed EduMPI Suite, which provides near-real-time visualization of MPI communication in an interactive GUI, making performance analysis immediately accessible in educational settings. While our previous work focused on usability and classroom integration, this paper presents the technical architecture and performance evaluation of EduMPI Suite's measurement and data-management system. We couple EduMPI, an extended Open MPI fork with integrated performance measurement, with EduStore, a time-series database that processes performance-relevant events in near-real-time for visualization in EduMPI GUI. EduMPI leverages a binary-based ingestion layer for high-throughput data insertion, avoiding costly data conversions. Combined with TimescaleDB's optimized time-series processing, this design ensures continuous data availability with negligible delay. On-the-fly aggregation and indexing enable responsive queries and smooth, interactive visualization of communication traces. Our evaluation demonstrates that EduMPI Suite introduces less than 3.8% runtime overhead while maintaining query latencies below 0.09 s, enabling interactive performance analysis of MPI applications. Usability studies confirm that EduMPI Suite significantly reduces entry barriers for students and improves their ability to identify performance issues over conventional tools.

Parallel programming curricula encompass not only the development of parallel code and algorithm design but also emphasize efficiency, optimization, and performance analysis. To equip students with the skills necessary for writing efficient parallel code using message passing with MPI, practical experience on HPC environments is essential. Performance analysis tools assist in identifying issues such as load imbalances or bottlenecks. Despite their use by experienced developers, these tools' complexity and required knowledge of cluster architectures, resource management, MPI, and common parallel issues hinder their educational integration. To address these barriers, we developed EduMPI, a learning support tool designed to simplify cluster usage and performance analysis for students. EduMPI offers an intuitive GUI that automates program execution on clusters and delivers near-real-time visualizations of MPI communication. This enables students to track process communication according to their physical placement within the cluster and detect performance problems interactively. This paper presents a user study comparing EduMPI with established professional performance analysis tools, demonstrating that EduMPI lowers entry barriers and fosters an intuitive understanding of parallel program performance, thereby enhancing its educational value.

In-depth understanding of collective MPI communication is a major challenge for both beginners and experienced developers. It is challenging to grasp the operations of collective algorithms, as many performance analysis tools cannot break down collective operations into their underlying point-to-point communication. However, collective communication is a key factor in optimizing the performance of parallel programs. EduMPI is a novel tool for parallel programming education, providing near-real-time visualization of collective MPI algorithms. It displays per-process data exchange and highlights performance issues like Late Sender and Late Receiver. This paper introduces the visualization of collective communication in EduMPI, demonstrates how this representation aids in understanding the concept of collective communication and its relevance for performance optimization, and evaluates its effectiveness in an educational context. EduMPI bridges the gap in understanding complex collective MPI operations by providing a transparent view of the underlying processes, enabling students to visualize and better understand the communication flow.

The use of high-performance clusters in parallel programming education can be challenging. Learners need prior knowledge of SSH, Linux commands, and workload management to execute their MPI programs on a cluster. Additionally, performance analysis, which is crucial for improving program behavior and identifying errors, requires an understanding of instrumentation, measurement, and typical performance issues in parallel programs. This requirement impedes beginners from benefiting from performance visualization. To make these instruments available to beginners in parallel programming, we are developing EduMPI. This novel learning support tool allows learners to execute MPI programs in a real environment on high-performance clusters from an early stage of parallel programming education. EduMPI helps learners to gain visual insights into their program’s performance through analysis at runtime.

The use of high-performance clusters in parallel programming education can be challenging. Learners need prior knowledge of SSH, Linux commands, and workload management to execute their MPI programs on a cluster. Additionally, performance analysis, which is crucial for improving program behavior and identifying errors, requires an understanding of instrumentation, measurement, and typical performance issues in parallel programs. This requirement impedes beginners from benefiting from performance visualization. To make these instruments available to beginners in parallel programming, we are developing EduMPI. This novel learning support tool allows learners to execute MPI programs in a real environment on high-performance clusters from an early stage of parallel programming education. EduMPI helps learners to gain visual insights into their program’s performance through analysis at runtime.

Performance analysis tools are frequently used to support the development of parallel MPI applications. They facilitate the detection of errors, bottlenecks, or inefficiencies but differ substantially in their instrumentation, measurement, and type of feedback. Especially, tools that provide visual feedback are helpful for educational purposes. They provide a visual abstraction of program behavior, supporting learners to identify and understand performance issues and write more efficient code. However, existing professional tools for performance analysis are very complex, and their use in beginner courses can be very demanding. Foremost, their instrumentation and measurement require deep knowledge and take a long time. Immediate, as well as straightforward feedback, is essential to motivate learners. This paper provides an extensive overview of performance analysis tools for parallel MPI applications, which experienced developers broadly use today. It also gives an overview of existing educational tools for parallel programming with MPI and shows their shortcomings compared to professional tools. Using tools for performance analysis of MPI programs in educational scenarios can promote the understanding of program behavior in large HPC systems and support learning parallel programming. At the same time, the complexity of the programs and the lack of infrastructure in educational institutions are barriers. These aspects will be considered and discussed in detail.