--- title: HPDC 2024 Digest venue: HPDC year: 2024 date: '2024-01-01' tags: [] paper_count: 12 draft: false --- 12 papers selected. --- ### Efficient all-to-all Collective Communication Schedules for Direct-connect Topologies *Prithwish Basu, Liangyu Zhao, Jason Fantl, Siddharth Pal *et al.** **TL;DR** — Derives near-optimal all-to-all collective communication schedules for direct-connect HPC topologies, directly improving bandwidth utilization in large-scale distributed systems. --- ### Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field *Isaac Boixaderas, Sergi Moré, Javier Bartolome, David Vicente *et al.** **TL;DR** — Applies reinforcement learning to dynamically mitigate uncorrected DRAM errors at production HPC scale, improving system reliability without sacrificing performance. --- ### IDT: Intelligent Data Placement for Multi-tiered Main Memory with Reinforcement Learning *Juneseo Chang, Wanju Doh, Yaebin Moon, Eojin Lee *et al.** **TL;DR** — Presents a reinforcement learning-driven runtime that automatically places data across heterogeneous memory tiers, reducing access latency in HPC nodes with complex memory hierarchies. --- ### FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example *Marcin Copik, Alexandru Calotoiu, Pengyu Zhou, Konstantin Taranov *et al.** **TL;DR** — Reconstructs ZooKeeper as a fully serverless service and distills concrete design lessons for building stateful distributed coordination primitives on FaaS platforms. --- ### ESG: Pipeline-Conscious Efficient Scheduling of DNN Workflows on Serverless Platforms with Shareable GPUs *Xinning Hui, Yuanchao Xu 0001, Zhishan Guo, Xipeng Shen* **TL;DR** — Introduces a pipeline-aware scheduler that shares GPUs across serverless DNN workflow stages, substantially cutting end-to-end latency and GPU idle time. --- ### FASOP: Fast yet Accurate Automated Search for Optimal Parallelization of Transformers on Heterogeneous GPU Clusters *Sunyeol Hwang, Eungyeong Lee, Hongseok Oh 0003, Youngmin Yi* **TL;DR** — Provides a fast, model-driven search strategy that finds optimal tensor/pipeline/data parallelism configurations for transformer training on heterogeneous GPU clusters. --- ### Near-Optimal Wafer-Scale Reduce *Piotr Luczynski, Lukas Gianinazzi, Patrick Iff, Leighton Wilson *et al.** **TL;DR** — Designs and analyzes near-optimal Reduce collective algorithms tailored to wafer-scale interconnect topology, setting new performance bounds for next-generation HPC hardware. --- ### DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models *Avinash Maurya, Robert Underwood, M. Mustafa Rafique, Franck Cappello *et al.** **TL;DR** — Introduces lazy asynchronous checkpointing that overlaps LLM training with I/O, dramatically reducing checkpoint overhead on large-scale HPC storage systems. --- ### ADTopk: All-Dimension Top-k Compression for High-Performance Data-Parallel DNN Training *Zhangqiang Ming, Yuchong Hu, Wenxiang Zhou, Xinjue Zheng *et al.** **TL;DR** — Proposes an all-dimension top-k gradient sparsification scheme that reduces communication volume in data-parallel distributed training while preserving convergence quality. --- ### Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems *Thanh Son Phung, Colin Thomas, Logan T. Ward, Kyle Chard *et al.** **TL;DR** — Introduces context reuse across scientific workflow tasks, allowing distributed workflow systems to cache and share intermediate computation artifacts and significantly reduce redundant work. --- ### CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2 *Shihui Song, Yafan Huang, Peng Jiang 0004, Xiaodong Yu 0001 *et al.** **TL;DR** — Ports and scales error-bounded lossy compression to the Cerebras CS-2 wafer-scale engine, enabling significant data reduction for HPC scientific workloads on novel accelerator hardware. --- ### EvoStore: Towards Scalable Storage of Evolving Learning Models *Robert Underwood, Meghana Madhyastha, Randal C. Burns, Bogdan Nicolae* **TL;DR** — Designs a storage system that efficiently manages the versioned, incrementally evolving checkpoints produced during large-scale distributed model training, reducing storage overhead and retrieval time.