AI Tools & Models HOTFeaturedAll
arXiv cs.AIScore65

ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding

Summary

ForeSpark jointly schedules draft and verification budgets for parallel drafters, speeding up LLM decoding when a whole block of draft tokens is proposed in one forward pass.

Why this matters

Covers LLM · Relevant to this topic

Open original ↗

Source: arXiv cs.AI · arxiv.org

Topics

Related items