Vincent Lannurien
d822cdaa6a
4.3 KiB
title, venue, year, date, tags, paper_count, draft
| title | venue | year | date | tags | paper_count | draft |
|---|---|---|---|---|---|---|
| HPDC 2024 Digest | HPDC | 2024 | 2024-01-01 | 12 | false |
12 papers selected.
Efficient all-to-all Collective Communication Schedules for Direct-connect Topologies
Prithwish Basu, Liangyu Zhao, Jason Fantl, Siddharth Pal et al.
TL;DR — Derives near-optimal all-to-all collective communication schedules for direct-connect HPC topologies, directly improving bandwidth utilization in large-scale distributed systems.
Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field
Isaac Boixaderas, Sergi Moré, Javier Bartolome, David Vicente et al.
TL;DR — Applies reinforcement learning to dynamically mitigate uncorrected DRAM errors at production HPC scale, improving system reliability without sacrificing performance.
IDT: Intelligent Data Placement for Multi-tiered Main Memory with Reinforcement Learning
Juneseo Chang, Wanju Doh, Yaebin Moon, Eojin Lee et al.
TL;DR — Presents a reinforcement learning-driven runtime that automatically places data across heterogeneous memory tiers, reducing access latency in HPC nodes with complex memory hierarchies.
FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example
Marcin Copik, Alexandru Calotoiu, Pengyu Zhou, Konstantin Taranov et al.
TL;DR — Reconstructs ZooKeeper as a fully serverless service and distills concrete design lessons for building stateful distributed coordination primitives on FaaS platforms.
ESG: Pipeline-Conscious Efficient Scheduling of DNN Workflows on Serverless Platforms with Shareable GPUs
Xinning Hui, Yuanchao Xu 0001, Zhishan Guo, Xipeng Shen
TL;DR — Introduces a pipeline-aware scheduler that shares GPUs across serverless DNN workflow stages, substantially cutting end-to-end latency and GPU idle time.
FASOP: Fast yet Accurate Automated Search for Optimal Parallelization of Transformers on Heterogeneous GPU Clusters
Sunyeol Hwang, Eungyeong Lee, Hongseok Oh 0003, Youngmin Yi
TL;DR — Provides a fast, model-driven search strategy that finds optimal tensor/pipeline/data parallelism configurations for transformer training on heterogeneous GPU clusters.
Near-Optimal Wafer-Scale Reduce
Piotr Luczynski, Lukas Gianinazzi, Patrick Iff, Leighton Wilson et al.
TL;DR — Designs and analyzes near-optimal Reduce collective algorithms tailored to wafer-scale interconnect topology, setting new performance bounds for next-generation HPC hardware.
DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models
Avinash Maurya, Robert Underwood, M. Mustafa Rafique, Franck Cappello et al.
TL;DR — Introduces lazy asynchronous checkpointing that overlaps LLM training with I/O, dramatically reducing checkpoint overhead on large-scale HPC storage systems.
ADTopk: All-Dimension Top-k Compression for High-Performance Data-Parallel DNN Training
Zhangqiang Ming, Yuchong Hu, Wenxiang Zhou, Xinjue Zheng et al.
TL;DR — Proposes an all-dimension top-k gradient sparsification scheme that reduces communication volume in data-parallel distributed training while preserving convergence quality.
Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems
Thanh Son Phung, Colin Thomas, Logan T. Ward, Kyle Chard et al.
TL;DR — Introduces context reuse across scientific workflow tasks, allowing distributed workflow systems to cache and share intermediate computation artifacts and significantly reduce redundant work.
CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2
Shihui Song, Yafan Huang, Peng Jiang 0004, Xiaodong Yu 0001 et al.
TL;DR — Ports and scales error-bounded lossy compression to the Cerebras CS-2 wafer-scale engine, enabling significant data reduction for HPC scientific workloads on novel accelerator hardware.
EvoStore: Towards Scalable Storage of Evolving Learning Models
Robert Underwood, Meghana Madhyastha, Randal C. Burns, Bogdan Nicolae
TL;DR — Designs a storage system that efficiently manages the versioned, incrementally evolving checkpoints produced during large-scale distributed model training, reducing storage overhead and retrieval time.