multi-topic, publish from gh-pages branch

site/content/cloud-edge/digests/OSDI-2024/index.md

@@ -0,0 +1,150 @@
+---
+title: OSDI 2024 Digest
+venue: OSDI
+year: 2024
+date: '2024-07-10'
+tags:
+- llm-serving
+- distributed-systems
+- verification
+- memory
+- networking
+- storage
+draft: false
+paper_count: 11
+---
+
+11 papers selected.
+
+---
+
+### DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
+
+*Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu *et al.**
+
+**TL;DR** — Separates the compute-heavy prefill phase from the memory-bound decoding phase onto different GPU pools, eliminating head-of-line blocking and significantly improving LLM serving throughput.
+
+**Why notable** — Became one of the most influential LLM systems papers of 2024; the prefill–decode disaggregation insight is now widely adopted in production inference stacks (vLLM, SGLang, etc.).
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin)
+
+---
+
+### Fairness in Serving Large Language Models
+
+*Ying Sheng 0007, Shiyi Cao, Dacheng Li, Banghua Zhu *et al.**
+
+**TL;DR** — Introduces VTC, a token-count-weighted fair scheduling policy that prevents long-prompt users from monopolising GPU capacity in multi-tenant LLM services.
+
+**Why notable** — First paper to formally study multi-tenant fairness in LLM serving; directly influenced subsequent work on SLA-aware serving and resource allocation in shared inference clusters.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sheng)
+
+---
+
+### Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
+
+*Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan *et al.**
+
+**TL;DR** — Introduces chunked prefill and stall-free scheduling to decouple throughput and latency goals, letting the same serving system meet both SLOs simultaneously.
+
+**Why notable** — Elegant framing of the throughput–latency tension; chunked prefill became a standard technique in open-source inference engines within months of publication.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/agrawal)
+
+---
+
+### Llumnix: Dynamic Scheduling for Large Language Model Serving
+
+*Biao Sun 0002, Ziming Huang, Hanyu Zhao, Wencong Xiao *et al.**
+
+**TL;DR** — Treats in-flight LLM requests as migratable units, enabling load balancing and SLO recovery by live-migrating KV-cache state across GPU instances.
+
+**Why notable** — Request migration for LLM serving was considered impractical due to KV-cache size; this paper shows it is feasible and impactful, opening a new design dimension for inference schedulers.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sun-biao)
+
+---
+
+### MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale
+
+*Arnab Choudhury, Yang Wang 0009, Tuomas Pelkonen, Kutta Srinivasan *et al.**
+
+**TL;DR** — Describes Google's production system for scheduling ML training jobs across geographically distributed datacenters, balancing GPU utilisation, job deadlines, and cross-datacenter bandwidth costs.
+
+**Why notable** — Rare large-scale production paper on global ML scheduling; the insights on heterogeneous cluster management and placement constraints are directly useful for anyone operating multi-site GPU infrastructure.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/choudhury)
+
+---
+
+### SquirrelFS: using the Rust compiler to check file-system crash consistency
+
+*Hayley LeBlanc, Nathan Taylor, James Bornholt, Vijay Chidambaram*
+
+**TL;DR** — Encodes crash-consistency invariants in Rust's type system so that a file system that compiles is guaranteed not to leave the storage in an inconsistent state after a crash.
+
+**Why notable** — A clean demonstration that language-level type checking can replace runtime or proof-assistant-based verification for an important systems property; the approach is general and practically viable.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/leblanc)
+
+---
+
+### Anvil: Verifying Liveness of Cluster Management Controllers
+
+*Xudong Sun 0013, Wenjie Ma, Jiawei Tyler Gu, Zicheng Ma *et al.**
+
+**TL;DR** — Presents the first framework for mechanically verifying liveness (eventual progress) of Kubernetes-style reconciliation controllers, with proofs for real controllers including ZooKeeper and RabbitMQ operators.
+
+**Why notable** — Liveness proofs for real-world cloud controllers were previously out of reach; Anvil's methodology closes a critical gap in the formal verification of cloud infrastructure.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/sun-xudong)
+
+---
+
+### DRust: Language-Guided Distributed Shared Memory with Fine Granularity, Full Transparency, and Ultra Efficiency
+
+*Haoran Ma, Yifan Qiao 0002, Shi Liu, Shan Yu *et al.**
+
+**TL;DR** — Exploits Rust's ownership model to implement distributed shared memory at cache-line granularity, achieving near-local performance with no programmer annotations.
+
+**Why notable** — Prior DSM systems required explicit data placement or suffered high coherence overhead; DRust shows that a language's ownership semantics can serve as a zero-overhead coherence protocol.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/ma-haoran)
+
+---
+
+### Nomad: Non-Exclusive Memory Tiering via Transactional Page Migration
+
+*Lingfeng Xiang, Zhen Lin, Weishu Deng, Hui Lu 0001 *et al.**
+
+**TL;DR** — Enables tiered-memory systems to migrate pages concurrently with ongoing accesses using a transactional protocol, eliminating the stop-the-world pauses of existing page-migration approaches.
+
+**Why notable** — CXL-based memory tiering is becoming essential for cost-effective cloud deployments; Nomad's non-exclusive migration is a key enabling mechanism for practical tiering at scale.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/xiang)
+
+---
+
+### Fast and Scalable In-network Lock Management Using Lock Fission
+
+*Hanze Zhang, Ke Cheng, Rong Chen 0001, Haibo Chen 0001*
+
+**TL;DR** — Splits a distributed lock into independent sub-locks held in programmable switches, allowing lock acquisition to complete in a single network round-trip without touching any server CPU.
+
+**Why notable** — Achieves latencies previously only possible with RDMA using commodity programmable switching hardware; the lock-fission abstraction generalises cleanly to other in-network coordination primitives.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/zhang-hanze)
+
+---
+
+### Chop Chop: Byzantine Atomic Broadcast to the Network Limit
+
+*Martina Camaioni, Rachid Guerraoui, Matteo Monti, Pierre-Louis Roman *et al.**
+
+**TL;DR** — Achieves Byzantine fault-tolerant atomic broadcast at near-network-bandwidth rates by batching, pipelining, and carefully overlapping cryptographic operations with network I/O.
+
+**Why notable** — Closes the gap between the theoretical throughput of BFT protocols and what commodity hardware can actually deliver; relevant baseline for any production BFT system design.
+
+[→ Read paper](https://www.usenix.org/conference/osdi24/presentation/camaioni)
+