159 lines
7.9 KiB
Markdown
159 lines
7.9 KiB
Markdown
---
|
|
title: SoCC 2024 Digest
|
|
venue: SoCC
|
|
year: 2024
|
|
date: '2024-11-01'
|
|
tags:
|
|
- cloud-computing
|
|
- distributed-systems
|
|
paper_count: 12
|
|
draft: false
|
|
---
|
|
|
|
12 papers selected.
|
|
|
|
---
|
|
|
|
### Queue Management for SLO-Oriented Large Language Model Serving
|
|
|
|
*Archit Patke, Dhemath Reddy, Saurabh Jha, Haoran Qiu *et al.**
|
|
|
|
**TL;DR** — A queue management framework that enforces latency SLOs for LLM serving by dynamically routing and prioritizing requests across heterogeneous inference capacity.
|
|
|
|
**Why notable** — As LLM deployments move into production clouds, meeting strict time-to-first-token and total latency SLOs becomes critical; this work directly addresses that gap with a practical, deployable solution. It is one of the first papers to treat LLM serving as a cloud SLO-management problem rather than a pure model-optimization problem.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698523)
|
|
|
|
---
|
|
|
|
### Kale: Elastic GPU Scheduling for Online DL Model Training
|
|
|
|
*Ziyang Liu, Renyu Yang, Jin Ouyang, Weihan Jiang *et al.**
|
|
|
|
**TL;DR** — Kale elastically resizes GPU allocations for online DL training jobs in response to real-time resource pressure, improving cluster utilization without violating training progress guarantees.
|
|
|
|
**Why notable** — Elastic GPU scheduling is an unsolved pain point in shared ML clusters; Kale's ability to dynamically shrink and expand jobs without checkpointing overhead is directly applicable to production training infrastructure at hyperscalers.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698532)
|
|
|
|
---
|
|
|
|
### Hops: Fine-grained heterogeneous sensing, efficient and fair Deep Learning cluster scheduling system
|
|
|
|
*Qinghe Wang, Futian Wang, Xinwei Zheng*
|
|
|
|
**TL;DR** — Hops uses fine-grained, heterogeneity-aware GPU sensing to make scheduling decisions that are simultaneously efficient and fair across diverse DL workloads.
|
|
|
|
**Why notable** — Hardware heterogeneity in GPU clusters is the norm, not the exception; Hops provides a principled framework for exploiting that diversity, making it immediately relevant to operators of mixed-generation GPU fleets.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698515)
|
|
|
|
---
|
|
|
|
### Process-as-a-Service: Unifying Elastic and Stateful Clouds with Serverless Processes
|
|
|
|
*Marcin Copik, Alexandru Calotoiu, Gyorgy Réthy, Roman Böhringer *et al.**
|
|
|
|
**TL;DR** — PraaS introduces a long-lived, stateful serverless process abstraction that bridges the gap between ephemeral FaaS functions and persistent cloud VMs.
|
|
|
|
**Why notable** — Statelessness is the central limitation of today's FaaS platforms; this paper proposes a well-grounded new programming model that could reshape how developers think about serverless, backed by implementation and evaluation at scale.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698567)
|
|
|
|
---
|
|
|
|
### FaPES: Enabling Efficient Elastic Scaling for Serverless Machine Learning Platforms
|
|
|
|
*Xiaoyang Zhao 0005, Siran Yang, Jiamang Wang, Lansong Diao *et al.**
|
|
|
|
**TL;DR** — FaPES achieves fast, fine-grained vertical and horizontal scaling of serverless ML serving pods by decoupling memory provisioning from compute allocation.
|
|
|
|
**Why notable** — Elastic scaling for ML inference is a key cost driver in cloud ML platforms; FaPES demonstrates sub-second scaling decisions that reduce both cold-start overhead and resource waste, with results from a production deployment.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698548)
|
|
|
|
---
|
|
|
|
### Faascale: Scaling MicroVM Vertically for Serverless Computing with Memory Elasticity
|
|
|
|
*Xinmin Zhang, Qiang He 0001, Hao Fan 0006, Song Wu 0001*
|
|
|
|
**TL;DR** — Faascale enables runtime vertical memory scaling of Firecracker microVMs for serverless functions, eliminating the need to restart or pre-provision fixed memory sizes.
|
|
|
|
**Why notable** — Memory over-provisioning is a major cost inefficiency in serverless platforms; Faascale's live memory elasticity directly reduces waste while maintaining the isolation guarantees of microVM-based FaaS.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698512)
|
|
|
|
---
|
|
|
|
### AutoBurst: Autoscaling Burstable Instances for Cost-effective Latency SLOs
|
|
|
|
*Rubaba Hasan, Timothy Zhu, Bhuvan Urgaonkar*
|
|
|
|
**TL;DR** — AutoBurst exploits burstable cloud instance types and their CPU credit mechanics to autoscale services at lower cost while still meeting tail-latency SLOs.
|
|
|
|
**Why notable** — Burstable instances are widely available on all major clouds yet poorly understood for SLO-sensitive workloads; this paper provides a rigorous autoscaling policy that unlocks significant cost savings without sacrificing latency guarantees.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698530)
|
|
|
|
---
|
|
|
|
### Dynamic Idle Resource Leasing To Safely Oversubscribe Capacity At Meta
|
|
|
|
*Nishant Gupta, Iyswarya Narayanan, Shivam Handa, Sayak Chakraborti *et al.**
|
|
|
|
**TL;DR** — Meta's production system dynamically lends idle reserved-capacity to opportunistic workloads, recovering stranded compute while ensuring low-latency eviction when owners reclaim resources.
|
|
|
|
**Why notable** — This industry paper provides rare visibility into hyperscale capacity management at Meta's scale, demonstrating that safe oversubscription can recover tens of percent of otherwise idle datacenter capacity.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698537)
|
|
|
|
---
|
|
|
|
### Forecasting Algorithms for Intelligent Resource Scaling: An Experimental Analysis
|
|
|
|
*Yanlei Diao, Dominik Horn, Andreas Kipf, Oleksandr Shchur *et al.**
|
|
|
|
**TL;DR** — A comprehensive empirical study comparing classical and learned forecasting algorithms for cloud autoscaling, yielding concrete guidelines on when each approach wins.
|
|
|
|
**Why notable** — Autoscaling relies heavily on workload forecasting, yet practitioners lack principled guidance on algorithm choice; this work from the MIT/AWS group fills that gap with rigorous experimentation across real-world cloud traces.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698564)
|
|
|
|
---
|
|
|
|
### Vista: Machine Learning based Database Performance Troubleshooting Framework in Amazon RDS
|
|
|
|
*Vikramank Y. Singh, Zhao Song 0001, Balakrishnan (Murali) Narayanaswamy, Kapil Eknath Vaidya *et al.**
|
|
|
|
**TL;DR** — Vista is a production ML framework deployed in Amazon RDS that automatically diagnoses performance regressions by correlating database metrics with causal performance models.
|
|
|
|
**Why notable** — Database performance debugging at cloud scale is labor-intensive and error-prone; Vista's deployment in RDS demonstrates how ML-driven root-cause analysis can reduce mean-time-to-resolution for thousands of customer instances.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698519)
|
|
|
|
---
|
|
|
|
### Inshrinkerator: Compressing Deep Learning Training Checkpoints via Dynamic Quantization
|
|
|
|
*Amey Agrawal, Sameer Reddy, Satwik Bhattamishra, Venkata Prabhakara Sarath Nookala *et al.**
|
|
|
|
**TL;DR** — Inshrinkerator applies dynamic quantization to DL training checkpoints at save time, reducing checkpoint sizes by up to 4x with negligible impact on training convergence.
|
|
|
|
**Why notable** — Checkpoint storage and I/O are significant costs in large-scale distributed training; this work provides a transparent, easy-to-adopt compression layer that can be retrofitted into existing training pipelines.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698553)
|
|
|
|
---
|
|
|
|
### The Sunk Carbon Fallacy: Rethinking Carbon Footprint Metrics for Effective Carbon-Aware Scheduling
|
|
|
|
*Noman Bashir, Varun Gohil, Anagha Belavadi Subramanya, Mohammad Shahrad *et al.**
|
|
|
|
**TL;DR** — The paper argues that conventional carbon metrics misattribute embodied (manufacturing) carbon as a fixed sunk cost, and proposes revised metrics that make carbon-aware scheduling decisions more accurate and actionable.
|
|
|
|
**Why notable** — Carbon-aware cloud scheduling is an emerging priority, but flawed metrics can lead to counterproductive decisions; this work from the Delimitrou and Irwin groups provides a conceptual correction with broad implications for green cloud policy and tooling.
|
|
|
|
[→ Read paper](https://doi.org/10.1145/3698038.3698542)
|
|
|