8484abea47
Hugo/PaperMod static site tracking 13 conferences and 7 journals for edge and cloud systems research. Includes FullCalendar deadline view, ICORE/SCImago rankings, DBLP paper digest pipeline, and Python fetch/generate scripts. PaperMod added as a git submodule.
151 lines
7.0 KiB
Markdown
151 lines
7.0 KiB
Markdown
---
|
||
title: OSDI 2024 Digest
|
||
venue: OSDI
|
||
year: 2024
|
||
date: '2024-07-10'
|
||
tags:
|
||
- llm-serving
|
||
- distributed-systems
|
||
- verification
|
||
- memory
|
||
- networking
|
||
- storage
|
||
draft: false
|
||
paper_count: 11
|
||
---
|
||
|
||
11 papers selected.
|
||
|
||
---
|
||
|
||
### DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
|
||
|
||
*Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu *et al.**
|
||
|
||
**TL;DR** — Separates the compute-heavy prefill phase from the memory-bound decoding phase onto different GPU pools, eliminating head-of-line blocking and significantly improving LLM serving throughput.
|
||
|
||
**Why notable** — Became one of the most influential LLM systems papers of 2024; the prefill–decode disaggregation insight is now widely adopted in production inference stacks (vLLM, SGLang, etc.).
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin)
|
||
|
||
---
|
||
|
||
### Fairness in Serving Large Language Models
|
||
|
||
*Ying Sheng 0007, Shiyi Cao, Dacheng Li, Banghua Zhu *et al.**
|
||
|
||
**TL;DR** — Introduces VTC, a token-count-weighted fair scheduling policy that prevents long-prompt users from monopolising GPU capacity in multi-tenant LLM services.
|
||
|
||
**Why notable** — First paper to formally study multi-tenant fairness in LLM serving; directly influenced subsequent work on SLA-aware serving and resource allocation in shared inference clusters.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sheng)
|
||
|
||
---
|
||
|
||
### Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
|
||
|
||
*Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan *et al.**
|
||
|
||
**TL;DR** — Introduces chunked prefill and stall-free scheduling to decouple throughput and latency goals, letting the same serving system meet both SLOs simultaneously.
|
||
|
||
**Why notable** — Elegant framing of the throughput–latency tension; chunked prefill became a standard technique in open-source inference engines within months of publication.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/agrawal)
|
||
|
||
---
|
||
|
||
### Llumnix: Dynamic Scheduling for Large Language Model Serving
|
||
|
||
*Biao Sun 0002, Ziming Huang, Hanyu Zhao, Wencong Xiao *et al.**
|
||
|
||
**TL;DR** — Treats in-flight LLM requests as migratable units, enabling load balancing and SLO recovery by live-migrating KV-cache state across GPU instances.
|
||
|
||
**Why notable** — Request migration for LLM serving was considered impractical due to KV-cache size; this paper shows it is feasible and impactful, opening a new design dimension for inference schedulers.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sun-biao)
|
||
|
||
---
|
||
|
||
### MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale
|
||
|
||
*Arnab Choudhury, Yang Wang 0009, Tuomas Pelkonen, Kutta Srinivasan *et al.**
|
||
|
||
**TL;DR** — Describes Google's production system for scheduling ML training jobs across geographically distributed datacenters, balancing GPU utilisation, job deadlines, and cross-datacenter bandwidth costs.
|
||
|
||
**Why notable** — Rare large-scale production paper on global ML scheduling; the insights on heterogeneous cluster management and placement constraints are directly useful for anyone operating multi-site GPU infrastructure.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/choudhury)
|
||
|
||
---
|
||
|
||
### SquirrelFS: using the Rust compiler to check file-system crash consistency
|
||
|
||
*Hayley LeBlanc, Nathan Taylor, James Bornholt, Vijay Chidambaram*
|
||
|
||
**TL;DR** — Encodes crash-consistency invariants in Rust's type system so that a file system that compiles is guaranteed not to leave the storage in an inconsistent state after a crash.
|
||
|
||
**Why notable** — A clean demonstration that language-level type checking can replace runtime or proof-assistant-based verification for an important systems property; the approach is general and practically viable.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/leblanc)
|
||
|
||
---
|
||
|
||
### Anvil: Verifying Liveness of Cluster Management Controllers
|
||
|
||
*Xudong Sun 0013, Wenjie Ma, Jiawei Tyler Gu, Zicheng Ma *et al.**
|
||
|
||
**TL;DR** — Presents the first framework for mechanically verifying liveness (eventual progress) of Kubernetes-style reconciliation controllers, with proofs for real controllers including ZooKeeper and RabbitMQ operators.
|
||
|
||
**Why notable** — Liveness proofs for real-world cloud controllers were previously out of reach; Anvil's methodology closes a critical gap in the formal verification of cloud infrastructure.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sun-xudong)
|
||
|
||
---
|
||
|
||
### DRust: Language-Guided Distributed Shared Memory with Fine Granularity, Full Transparency, and Ultra Efficiency
|
||
|
||
*Haoran Ma, Yifan Qiao 0002, Shi Liu, Shan Yu *et al.**
|
||
|
||
**TL;DR** — Exploits Rust's ownership model to implement distributed shared memory at cache-line granularity, achieving near-local performance with no programmer annotations.
|
||
|
||
**Why notable** — Prior DSM systems required explicit data placement or suffered high coherence overhead; DRust shows that a language's ownership semantics can serve as a zero-overhead coherence protocol.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/ma-haoran)
|
||
|
||
---
|
||
|
||
### Nomad: Non-Exclusive Memory Tiering via Transactional Page Migration
|
||
|
||
*Lingfeng Xiang, Zhen Lin, Weishu Deng, Hui Lu 0001 *et al.**
|
||
|
||
**TL;DR** — Enables tiered-memory systems to migrate pages concurrently with ongoing accesses using a transactional protocol, eliminating the stop-the-world pauses of existing page-migration approaches.
|
||
|
||
**Why notable** — CXL-based memory tiering is becoming essential for cost-effective cloud deployments; Nomad's non-exclusive migration is a key enabling mechanism for practical tiering at scale.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/xiang)
|
||
|
||
---
|
||
|
||
### Fast and Scalable In-network Lock Management Using Lock Fission
|
||
|
||
*Hanze Zhang, Ke Cheng, Rong Chen 0001, Haibo Chen 0001*
|
||
|
||
**TL;DR** — Splits a distributed lock into independent sub-locks held in programmable switches, allowing lock acquisition to complete in a single network round-trip without touching any server CPU.
|
||
|
||
**Why notable** — Achieves latencies previously only possible with RDMA using commodity programmable switching hardware; the lock-fission abstraction generalises cleanly to other in-network coordination primitives.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/zhang-hanze)
|
||
|
||
---
|
||
|
||
### Chop Chop: Byzantine Atomic Broadcast to the Network Limit
|
||
|
||
*Martina Camaioni, Rachid Guerraoui, Matteo Monti, Pierre-Louis Roman *et al.**
|
||
|
||
**TL;DR** — Achieves Byzantine fault-tolerant atomic broadcast at near-network-bandwidth rates by batching, pipelining, and carefully overlapping cryptographic operations with network I/O.
|
||
|
||
**Why notable** — Closes the gap between the theoretical throughput of BFT protocols and what commodity hardware can actually deliver; relevant baseline for any production BFT system design.
|
||
|
||
[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/camaioni)
|
||
|