109 lines
4.3 KiB
Markdown
109 lines
4.3 KiB
Markdown
---
|
|
title: HPDC 2024 Digest
|
|
venue: HPDC
|
|
year: 2024
|
|
date: '2024-01-01'
|
|
tags: []
|
|
paper_count: 12
|
|
draft: false
|
|
---
|
|
|
|
12 papers selected.
|
|
|
|
---
|
|
|
|
### Efficient all-to-all Collective Communication Schedules for Direct-connect Topologies
|
|
|
|
*Prithwish Basu, Liangyu Zhao, Jason Fantl, Siddharth Pal *et al.**
|
|
|
|
**TL;DR** — Derives near-optimal all-to-all collective communication schedules for direct-connect HPC topologies, directly improving bandwidth utilization in large-scale distributed systems.
|
|
|
|
---
|
|
|
|
### Reinforcement Learning-based Adaptive Mitigation of Uncorrected DRAM Errors in the Field
|
|
|
|
*Isaac Boixaderas, Sergi Moré, Javier Bartolome, David Vicente *et al.**
|
|
|
|
**TL;DR** — Applies reinforcement learning to dynamically mitigate uncorrected DRAM errors at production HPC scale, improving system reliability without sacrificing performance.
|
|
|
|
---
|
|
|
|
### IDT: Intelligent Data Placement for Multi-tiered Main Memory with Reinforcement Learning
|
|
|
|
*Juneseo Chang, Wanju Doh, Yaebin Moon, Eojin Lee *et al.**
|
|
|
|
**TL;DR** — Presents a reinforcement learning-driven runtime that automatically places data across heterogeneous memory tiers, reducing access latency in HPC nodes with complex memory hierarchies.
|
|
|
|
---
|
|
|
|
### FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example
|
|
|
|
*Marcin Copik, Alexandru Calotoiu, Pengyu Zhou, Konstantin Taranov *et al.**
|
|
|
|
**TL;DR** — Reconstructs ZooKeeper as a fully serverless service and distills concrete design lessons for building stateful distributed coordination primitives on FaaS platforms.
|
|
|
|
---
|
|
|
|
### ESG: Pipeline-Conscious Efficient Scheduling of DNN Workflows on Serverless Platforms with Shareable GPUs
|
|
|
|
*Xinning Hui, Yuanchao Xu 0001, Zhishan Guo, Xipeng Shen*
|
|
|
|
**TL;DR** — Introduces a pipeline-aware scheduler that shares GPUs across serverless DNN workflow stages, substantially cutting end-to-end latency and GPU idle time.
|
|
|
|
---
|
|
|
|
### FASOP: Fast yet Accurate Automated Search for Optimal Parallelization of Transformers on Heterogeneous GPU Clusters
|
|
|
|
*Sunyeol Hwang, Eungyeong Lee, Hongseok Oh 0003, Youngmin Yi*
|
|
|
|
**TL;DR** — Provides a fast, model-driven search strategy that finds optimal tensor/pipeline/data parallelism configurations for transformer training on heterogeneous GPU clusters.
|
|
|
|
---
|
|
|
|
### Near-Optimal Wafer-Scale Reduce
|
|
|
|
*Piotr Luczynski, Lukas Gianinazzi, Patrick Iff, Leighton Wilson *et al.**
|
|
|
|
**TL;DR** — Designs and analyzes near-optimal Reduce collective algorithms tailored to wafer-scale interconnect topology, setting new performance bounds for next-generation HPC hardware.
|
|
|
|
---
|
|
|
|
### DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models
|
|
|
|
*Avinash Maurya, Robert Underwood, M. Mustafa Rafique, Franck Cappello *et al.**
|
|
|
|
**TL;DR** — Introduces lazy asynchronous checkpointing that overlaps LLM training with I/O, dramatically reducing checkpoint overhead on large-scale HPC storage systems.
|
|
|
|
---
|
|
|
|
### ADTopk: All-Dimension Top-k Compression for High-Performance Data-Parallel DNN Training
|
|
|
|
*Zhangqiang Ming, Yuchong Hu, Wenxiang Zhou, Xinjue Zheng *et al.**
|
|
|
|
**TL;DR** — Proposes an all-dimension top-k gradient sparsification scheme that reduces communication volume in data-parallel distributed training while preserving convergence quality.
|
|
|
|
---
|
|
|
|
### Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems
|
|
|
|
*Thanh Son Phung, Colin Thomas, Logan T. Ward, Kyle Chard *et al.**
|
|
|
|
**TL;DR** — Introduces context reuse across scientific workflow tasks, allowing distributed workflow systems to cache and share intermediate computation artifacts and significantly reduce redundant work.
|
|
|
|
---
|
|
|
|
### CereSZ: Enabling and Scaling Error-bounded Lossy Compression on Cerebras CS-2
|
|
|
|
*Shihui Song, Yafan Huang, Peng Jiang 0004, Xiaodong Yu 0001 *et al.**
|
|
|
|
**TL;DR** — Ports and scales error-bounded lossy compression to the Cerebras CS-2 wafer-scale engine, enabling significant data reduction for HPC scientific workloads on novel accelerator hardware.
|
|
|
|
---
|
|
|
|
### EvoStore: Towards Scalable Storage of Evolving Learning Models
|
|
|
|
*Robert Underwood, Meghana Madhyastha, Randal C. Burns, Bogdan Nicolae*
|
|
|
|
**TL;DR** — Designs a storage system that efficiently manages the versioned, incrementally evolving checkpoints produced during large-scale distributed model training, reducing storage overhead and retrieval time.
|
|
|