Cloud-Computing

12 papers selected. Queue Management for SLO-Oriented Large Language Model Serving Archit Patke, Dhemath Reddy, Saurabh Jha, Haoran Qiu et al. TL;DR — A queue management framework that enforces latency SLOs for LLM serving by dynamically routing and prioritizing requests across heterogeneous inference capacity. Why notable — As LLM deployments move into production clouds, meeting strict time-to-first-token and total latency SLOs becomes critical; this work directly addresses that gap with a practical, deployable solution. It is one of the first papers to treat LLM serving as a cloud SLO-management problem rather than a pure model-optimization problem. ...

10 papers selected. Fair, Efficient Multi-Resource Scheduling for Stateless Serverless Functions with Anubis Amit Samanta 0001, Ryan Stutsman TL;DR — Anubis introduces a fair, multi-resource scheduler for stateless serverless functions that achieves efficiency without sacrificing isolation between tenants. Why notable — Fairness in serverless resource allocation is an open problem as functions compete for heterogeneous resources (CPU, memory, I/O); Anubis provides a concrete, deployable answer. The work directly addresses a gap in production FaaS platforms where existing schedulers optimize for throughput but ignore per-tenant equity. ...