8484abea47
Hugo/PaperMod static site tracking 13 conferences and 7 journals for edge and cloud systems research. Includes FullCalendar deadline view, ICORE/SCImago rankings, DBLP paper digest pipeline, and Python fetch/generate scripts. PaperMod added as a git submodule.
7.0 KiB
title, venue, year, date, tags, draft, paper_count
| title | venue | year | date | tags | draft | paper_count | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OSDI 2024 Digest | OSDI | 2024 | 2024-07-10 |
|
false | 11 |
11 papers selected.
DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu et al.
TL;DR — Separates the compute-heavy prefill phase from the memory-bound decoding phase onto different GPU pools, eliminating head-of-line blocking and significantly improving LLM serving throughput.
Why notable — Became one of the most influential LLM systems papers of 2024; the prefill–decode disaggregation insight is now widely adopted in production inference stacks (vLLM, SGLang, etc.).
Fairness in Serving Large Language Models
Ying Sheng 0007, Shiyi Cao, Dacheng Li, Banghua Zhu et al.
TL;DR — Introduces VTC, a token-count-weighted fair scheduling policy that prevents long-prompt users from monopolising GPU capacity in multi-tenant LLM services.
Why notable — First paper to formally study multi-tenant fairness in LLM serving; directly influenced subsequent work on SLA-aware serving and resource allocation in shared inference clusters.
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan et al.
TL;DR — Introduces chunked prefill and stall-free scheduling to decouple throughput and latency goals, letting the same serving system meet both SLOs simultaneously.
Why notable — Elegant framing of the throughput–latency tension; chunked prefill became a standard technique in open-source inference engines within months of publication.
Llumnix: Dynamic Scheduling for Large Language Model Serving
Biao Sun 0002, Ziming Huang, Hanyu Zhao, Wencong Xiao et al.
TL;DR — Treats in-flight LLM requests as migratable units, enabling load balancing and SLO recovery by live-migrating KV-cache state across GPU instances.
Why notable — Request migration for LLM serving was considered impractical due to KV-cache size; this paper shows it is feasible and impactful, opening a new design dimension for inference schedulers.
MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale
Arnab Choudhury, Yang Wang 0009, Tuomas Pelkonen, Kutta Srinivasan et al.
TL;DR — Describes Google's production system for scheduling ML training jobs across geographically distributed datacenters, balancing GPU utilisation, job deadlines, and cross-datacenter bandwidth costs.
Why notable — Rare large-scale production paper on global ML scheduling; the insights on heterogeneous cluster management and placement constraints are directly useful for anyone operating multi-site GPU infrastructure.
SquirrelFS: using the Rust compiler to check file-system crash consistency
Hayley LeBlanc, Nathan Taylor, James Bornholt, Vijay Chidambaram
TL;DR — Encodes crash-consistency invariants in Rust's type system so that a file system that compiles is guaranteed not to leave the storage in an inconsistent state after a crash.
Why notable — A clean demonstration that language-level type checking can replace runtime or proof-assistant-based verification for an important systems property; the approach is general and practically viable.
Anvil: Verifying Liveness of Cluster Management Controllers
Xudong Sun 0013, Wenjie Ma, Jiawei Tyler Gu, Zicheng Ma et al.
TL;DR — Presents the first framework for mechanically verifying liveness (eventual progress) of Kubernetes-style reconciliation controllers, with proofs for real controllers including ZooKeeper and RabbitMQ operators.
Why notable — Liveness proofs for real-world cloud controllers were previously out of reach; Anvil's methodology closes a critical gap in the formal verification of cloud infrastructure.
DRust: Language-Guided Distributed Shared Memory with Fine Granularity, Full Transparency, and Ultra Efficiency
Haoran Ma, Yifan Qiao 0002, Shi Liu, Shan Yu et al.
TL;DR — Exploits Rust's ownership model to implement distributed shared memory at cache-line granularity, achieving near-local performance with no programmer annotations.
Why notable — Prior DSM systems required explicit data placement or suffered high coherence overhead; DRust shows that a language's ownership semantics can serve as a zero-overhead coherence protocol.
Nomad: Non-Exclusive Memory Tiering via Transactional Page Migration
Lingfeng Xiang, Zhen Lin, Weishu Deng, Hui Lu 0001 et al.
TL;DR — Enables tiered-memory systems to migrate pages concurrently with ongoing accesses using a transactional protocol, eliminating the stop-the-world pauses of existing page-migration approaches.
Why notable — CXL-based memory tiering is becoming essential for cost-effective cloud deployments; Nomad's non-exclusive migration is a key enabling mechanism for practical tiering at scale.
Fast and Scalable In-network Lock Management Using Lock Fission
Hanze Zhang, Ke Cheng, Rong Chen 0001, Haibo Chen 0001
TL;DR — Splits a distributed lock into independent sub-locks held in programmable switches, allowing lock acquisition to complete in a single network round-trip without touching any server CPU.
Why notable — Achieves latencies previously only possible with RDMA using commodity programmable switching hardware; the lock-fission abstraction generalises cleanly to other in-network coordination primitives.
Chop Chop: Byzantine Atomic Broadcast to the Network Limit
Martina Camaioni, Rachid Guerraoui, Matteo Monti, Pierre-Louis Roman et al.
TL;DR — Achieves Byzantine fault-tolerant atomic broadcast at near-network-bandwidth rates by batching, pipelining, and carefully overlapping cryptographic operations with network I/O.
Why notable — Closes the gap between the theoretical throughput of BFT protocols and what commodity hardware can actually deliver; relevant baseline for any production BFT system design.