We proposed a new paradigm for serving LLMs with Service Level Objective guarantees. It is actively being integrated into mainstream LLM serving frameworks like SGLang PR Link. You can also check out the blog post here.