ViewTube

Red Hat's Mark Kurtz and Megan Flynn examine speculative decoding, a technique that uses a smaller, faster model—the ...

Speculative decoding is one of the most important performance optimizations in modern LLM serving—and most people still don't ...

This is a single lecture from a course. If you you like the material and want more context (e.g., the lectures that came before), check ...

Session covering an overview of speculative decoding and several seminal papers in the space, including Medusa, Eagle 1/2/3, ...

This video introduces the playlist, explains the Speculative Decoding Algorithm from the paper https://arxiv.org/pdf/2211.17192 in ...

Why is generating text with LLMs so slow? It's not a compute problem, it's a memory bandwidth problem. In this video, we explore ...

Speculative decoding is usually discussed as a way to make real time LLM APIs feel faster. But what happens when you apply it to ...

Modern LLM serving systems are built for scale, relying heavily on batching and parallelism to maximize GPU utilization. But as ...

Hello everyone! Host Ni-no is back, exploring a trending article from the archives that answers a burning question: 'Is there a ...

This video offers a formal and structured overview of the core ideas, methodology, and contributions of the work Conformal ...

Full Title：Yikai Zhu, Lukec Wang：SpecForge: Open Source Framework for Training Speculative Decoding Models Speculative ...

Learn how Speculative Decoding speeds up inference with a draft + verify approach. @zarabux.

Instead of using an autoregressive drafter in speculative decoding, it uses a discrete diffusion LLM that can draft whole token ...

... networking, scheduling, observability, training optimization, fused kernels, distillation, RL efficiency, speculative decoding, and ...

Notably, the architecture is shown to outperform both standard diffusion models and current state-of-the-art speculative decoding ...

... Selective Knowledge Distillation for Efficient Speculative Decoders' This work tackles inefficiencies in speculative decoding, ...

TiDAR outperforms speculative decoding in throughput and surpasses diffusion models like Dream and Llada in efficiency and ...

* Collaboration inquiries: commit.im@gmail.com (Please refrain from using personal emails.) The video animation was created ...

*Beating Speculative Decoding:* Why TiDAR offers a more efficient alternative to current acceleration methods like speculative ...

... scalable development, and optimization techniques like quantization and speculative decoding. Auto-scaling, batch processing, ...