ViewTube

One Click Templates Repo (free): https://github.com/TrelisResearch/one-click-llms Advanced Inference Repo (Paid Lifetime ...

Red Hat's Mark Kurtz and Megan Flynn examine speculative decoding, a technique that uses a smaller, faster model—the ...

About the seminar: https://faster-llms.vercel.app Speaker: Hongyang Zhang (Waterloo & Vector Institute) Title: EAGLE and ...

00:00 Introduction 01:15 Decoder-only inference 06:05 The KV cache 11:15 Continuous batching 16:17 Speculative decoding ...

Today, we're joined by Chris Lott, senior director of engineering at Qualcomm AI Research to discuss accelerating large language ...

There is a lot of possibility with Speculative Decoding allowing us normal folks to run larger and larger AI models at home. I hope ...

Abstract: We will discuss how vLLM combines continuous batching with speculative decoding with a focus on enabling external ...

Session covering an overview of speculative decoding and several seminal papers in the space, including Medusa, Eagle 1/2/3, ...

AI Gold Nugget #2.1 – Speaker #1: Raphael Vienne Can we go faster…

Speculative Architecture 14:24 Speculative Decoding Example 15:35 Introducing Medusa 16:53 Medusa's Decoding Heads 17:32 ...

In this vLLM office hours session, we explore the latest updates in vLLM v0.6.2, including Llama 3.2 Vision support, the ...

In this shot we implemented a cool paper - from zero to hero! Speculative decoding is a neat algorithm introduced by two different ...

We discussed the inference optimization technique known as Speculative Decoding with a world class researcher, expert, and ...

This talk explains why document parsing is still hard and why vision-first methods outperform OCR for complex layouts, merged ...

... vLLM core committer - An Intermediate Guide to Inference Using vLLM: PagedAttention, Quantization, Speculative Decoding, ...

In this video we discuss a technique called speculative decoding to speed up LLMs inference time. This has a number of use ...

... challenges behind them, including: Speculative Decoding, Prefix Caching, Disaggregated Prefill, and multi-accelerator support.

... downstream evaluation, and significant inference time speedups through techniques like speculative decoding and adaptive ...

Note for paper: Fast Inference from Transformers via Speculative Decoding (http://arxiv.org/abs/2211.17192v2) video slides: ...