CoPre-MoE: Consolidating and Preserving MoE for Multimodal Continual Instruction Tuning
Summary
This work lets multimodal LLMs sequentially acquire new capabilities without catastrophic forgetting, consolidating and preserving MoE experts.
Why this matters
Covers model and multimodal · Relevant to this topic
Source: arXiv cs.AI · arxiv.org