OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

arXiv:2607.231932026-07-27

In AI models that watch video and listen to audio at once, letting one sense decide for the other can throw away the answer

Omnimodal large language models that jointly process audio and video need to cut down the number of tokens (information pieces) to run in real time, but existing methods let one modality's importance scores decide what to keep in the other. This paper shows that for the same question, the moments that matter in audio and in video are often different, and proposes OmniScope, which lets each modality independently judge its own relevance to the query instead of one guiding the other. Without any additional training, it achieved the best average accuracy across compression settings on four benchmarks and two model sizes.

METAL MEDIA explanatory visual

OmniScope's three-stage compression pipeline

Evidence statusMeasured results reported

Query-aware scoringUsing the same question as a reference, video relevance is scored independently via CLIP and audio relevance via the model's own internal embeddings, then a token budget is split accordingly
Video pruning (AD-STC)Even-numbered frames keep overall scene information while odd-numbered frames keep only what changed from the previous frame, preserving both global context and temporal change
Audio mergingSimilar audio tokens within each one-second window are paired and averaged together, reducing redundancy while keeping temporal continuity
Performance validationAccuracy, speed, and memory were measured while retaining only 25-45% of tokens across four benchmarks and two model sizes, showing better results than competing methods

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Problem: existing omnimodal token compression methods use one modality's (often audio's) importance scores to decide what to keep in the other (video), but for the same question the moments that matter in audio and video often peak at different times, so this approach can discard information the answer actually needs.
Method: OmniScope uses the query as a shared anchor but lets audio and video independently compute their own relevance scores and split the token budget accordingly. On the video side, an anchor-delta strategy (AD-STC) alternates between frames that preserve overall scene context and frames that preserve only what changed over time; on the audio side, similar tokens within each one-second window are merged to reduce redundancy while keeping temporal continuity.
Testing: evaluated on four audio-video understanding benchmarks (WorldSense, DailyOmni, OmniVideoBench, Video-MME) and two sizes of the Qwen2.5-Omni model (7B and 3B) across multiple compression ratios.
Result: keeping only 45% of tokens caused almost no accuracy loss, and even at an aggressive 25% retention the 7B model's average accuracy dropped only 0.35 points versus using all tokens, compared to a 1.55-point drop for the competing OmniZip method. It also achieved up to 3.53x faster prefill (initial input processing) and more than 15% GPU memory savings.
Analysis: compression strengthens attention exchange across time windows and modalities; when one modality wrongly guides the other, this strengthened exchange gets steered toward answer-irrelevant cues, whereas letting each modality compress independently directs it toward genuinely query-relevant cues, as shown by attention visualizations.

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 0

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 1

Findings

At the aggressive 25% token retention, the 7B model's average accuracy dropped only 0.35 points versus the full-token baseline, compared to drops of 1.55 points for OmniZip, 0.82 for FastV, and 1.57 for random pruning.
At 45% retention, both the 7B and 3B models showed slight gains (+0.30, +0.03) or near-identical accuracy compared to using all tokens.
On the 7B model at 25% retention, prefill time dropped from 6299ms to 1784ms (3.53x speedup) and peak GPU memory dropped from 28.31G to 24.00G, over 15% reduction.
When independently sweeping video and audio compression ratios (Fig. 4), OmniScope showed the most gradual accuracy decline compared to all competing methods.
Replacing the query-based independent scoring with uniform compression on both modalities caused a 1.65-point average drop, and letting one modality guide the other caused roughly a 1.0-point drop, confirming the value of independent per-modality compression.

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 2

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 3

Where it can be used

Improving processing speed and memory use for services that must analyze audio and video together in real time, such as live video calls or streaming
Reducing inference cost for existing omnimodal models without retraining, for tasks like long-video question answering, content moderation, or video captioning
As a design reference for running audio-video models efficiently on resource-constrained (edge) devices

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 4

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 5

Limits and open work

Experiments are limited to the Qwen2.5-Omni model family (7B, 3B) and four specific benchmarks; behavior on other omnimodal model architectures is untested.
Results are measured with video input capped at 128 frames, so performance on much longer videos or other data conditions has not been separately verified.
Video-side scoring requires running a separate external CLIP model, adding a fixed extra cost that can be relatively larger for short-answer generation tasks.
The training-based competing method OmniSIFT could not be directly compared because its code was not publicly available at submission time.
The authors note that the model's internal audio-based scoring approach is not yet reliable enough to apply to the video side as well, so video scoring still depends on an external CLIP model.

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 10

OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models figure 11

Why it matters

Services that must understand sound and video together in real time, such as live video calls, video search, or content moderation, are limited by the growing number of tokens to process; OmniScope offers a practical, training-free design principle to reduce this cost. Because it does not rely on model-specific fine-tuning, the approach could be worth trying with other omnimodal models facing similar efficiency constraints.

Terms in this paper

Omnimodal large language model (OmniLLM) · an AI model that processes video, audio, and text together
Token compression · a technique for reducing the number of information pieces (tokens) a model must process, saving speed and memory
Query · the user's question, used here as the shared reference for deciding what to keep during compression
Anchor-Delta compression (AD-STC) · a video compression scheme where some frames keep overall scene information (anchor) and others keep only what changed (delta)
Prefill · the stage where the model processes the entire input once before generating an answer

Figures we cannot republish

Figure 6
Figure 7
Figure 8
Figure 9

See the figures in the original paper →

Original abstract (English)

Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a s

Authors · Jinsen Su

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Jinsen Su et al., arXiv:2607.23193, arxiv-nonexclusive