ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
Summary
ForeSpark jointly schedules draft and verification budgets for parallel drafters, speeding up LLM decoding when a whole block of draft tokens is proposed in one forward pass.
Why this matters
Covers LLM · Relevant to this topic
Source: arXiv cs.AI · arxiv.org