Figure 1: Illustration of AVE-Compass. Given the same editing instruction, the four panels illustrate characteristic ways current audio-visual editing models drift, with IF (Instruction Following) and FP (Fidelity Preserving) marks indicating where each output breaks.
Table 1: Comparison of AVE-Compass with existing editing benchmarks. Target Modality denotes the evaluated streams. Task Labels summarize task coverage with normalized labels, including object, scene, motion, camera, VFX, audio event, and speech. # Metrics and # Task Categories count evaluation dimensions and editing operation types, respectively. Cross-Modal Evaluation indicates whether the benchmark evaluates audio–video coupled editing, ranging from object-level correspondence to global-scene dependencies with diagnostic checks. Multi-Shot and Speech Edit indicate support for multi-shot source clips and speech-related editing. Difficulty Analysis denotes stratified evaluation over factors such as target localization, audio-source complexity, and cross-modal linkage. AVE-Compass is the only benchmark that jointly supports all these capabilities.
Benchmark
Target Modality
Target
Modality
Task Labels
Task
Labels
# Metrics
#
Metrics
# Task Categories
# Task
Categories
Cross-Modal Evaluation
Cross-Modal
Evaluation
Multi- Shot
Multi-
Shot
Speech Edit
Target
Modality
Task
Labels
#
Metrics
# Task
Categories
Cross-Modal
Evaluation
Multi-
Shot
Speech
Edit
Difficulty
Analysis
Video-Only Editing Benchmarks
VEBench Sun 2025
Video
Object, Scene
3
N/A
✗
✗
✗
✗
IVEBench Chen 2025
Video
Object, Scene
12
35
✗
✗
✗
✗
FiVE Li 2025
Video
Object
15
6
✗
✗
✗
✗
UniVBench Wei 2026
Video
Scene
21
6
✗
✓
✗
✗
CoVEBench Wu 2026
Video
Object, Scene
11
19
✗
✗
✗
✗
Audio-Visual Editing Benchmarks
SAVEBench Xu 2025
VideoAudio
Object
12
1
Object-level
✗
✗
✗
AVED-Bench Lin 2026
VideoAudio
Object
5
3
Object-level
✗
✗
✗
AVI-Edit Zheng 2025
VideoAudio
Audio Event
7
3
Object-level
✗
✗
✗
AVE-Compass (Ours)
VideoAudio
Object, Scene, Audio Event, Speech
Object, Scene,
Audio Event, Speech
19
28
Global Scene
✓
✓
✓
Object, Scene,
Audio Event, Speech
Figure 2: Dataset statistics of the AVE-Compass benchmark.
Table 2: MLLM-as-Judge results on AVE-Compass. We report four dimensions—Editing Intent, Instruction Following, Fidelity Preserving, and Realism—each as Overall / Video / Audio scores on a 0–100 scale. Models are ranked by the Overall Editing Intent score, which serves as the primary measure of complete edit execution by jointly accounting for instruction following and fidelity preservation. All metrics are higher-is-better. Best in bold. ∗Gemini misses 16 speech edits due to content moderation.
Model
Editing Intent
Instruction Following
Fidelity Preserving
Realism
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
AVE-Agent (Wan)
59.8+17.4
66.7+6.6
50.2+25.4
77.3+8.0
84.8+6.5
69.4+9.1
77.6+13.4
80.1+1.6
74.8+26.7
62.1+1.7
45.3+1.8
78.9+1.5
Wan2.7
42.4
60.1
24.8
69.3
78.3
60.3
64.2
78.5
48.1
60.4
43.5
77.4
HappyHorse
41.3
56.7
18.8
66.9
75.5
54.5
63.9
74.5
53.3
63.0
49.9
76.0
Gemini-Omni∗
38.0
56.1
10.0
44.9
74.0
10.8
84.9
76.8
96.1
66.9
49.2
84.5
Seedance
26.6
36.1
13.5
37.4
50.5
24.4
81.7
83.0
80.4
69.2
54.0
84.4
LTX2
15.2
10.7
26.4
70.1
72.8
66.2
30.6
24.5
42.3
64.1
42.0
86.2
Figure 3: Overview of AVE-Agent. Given a source video with audio and an edit instruction, the planner agent first preprocesses the input audio-visual clip and analyzes it through structured captioning. Conditioned on this analysis, it decomposes the instruction into a dependency-aware subtask DAG with per-step evaluation criteria. The executor agent executes the subtasks in order. For each subtask, it refines the intent into tool prompts and tool specifications, routes the request to the video, audio, or speech branch, and optimizes the result through a reflection loop guided by the planner-provided criteria. Finally, the mixed evaluator agent consumes the temporary assembled clip and planner criteria, scores instruction following, Fidelity Preserving, and quality, and outputs a control signal for final pass, remixing, subtask regeneration, or replanning.
Table 3: Automated metric results on AVE-Compass, grouped into Cross-Modal, Video, and Audio metrics. All metrics are higher-is-better. Best in bold. ∗Gemini misses 16 speech edits due to content moderation. †Speech Quality and Lip Sync are computed only on speech-category edits.
Model
Cross-Modal
Video
Audio
Lip Sync†
AV Sync
Video Aesthetic
Subject Consistency
Motion Smoothness
Audio Aesthetic
Speech Quality†
AVE-Agent (Wan)
0.622+0.091
0.766+0.073
0.452+0.001
0.972+0.003
0.987+0.001
0.614+0.005
0.368-0.060
Wan2.7
0.531
0.693
0.451
0.969
0.986
0.609
0.428
HappyHorse
0.620
0.695
0.439
0.975
0.989
0.650
0.726
Gemini-Omni∗
–
0.701
0.434
0.974
0.988
0.627
–
Seedance
0.431
0.718
0.430
0.971
0.987
0.629
0.388
LTX2
0.618
0.758
0.468
0.968
0.986
0.641
0.522
Figure 4: Robustness under difficulty. Left: Editing Intent across source-video duration. Right: object-localization hardness, audio-source complexity, and cross-modal linkage degree evaluated with modality-matched metrics. Hatched audio-source complexity bars mark Audio Fidelity Preserving inflated by audio non-response. Scores are on a 0–100 scale.
Table 4: Response-gated Fidelity Preserving and Realism by modality. Response rate measures whether the target modality changes. The “all” columns average over all evaluated cases, while the “gated” columns recompute Fidelity Preserving and Realism only on cases where the corresponding target modality responds. All metrics are higher-is-better. Bold marks the lowest, i.e., worst, response rates.
Model
Response Rate
Fidelity Preserving (all)
Fidelity Preserving (gated)
Realism (all)
Realism (gated)
Video
Audio
Video
Audio
Video
Audio
Video
Audio
Video
Audio
AVE-Agent (Wan)
92.7
94.4
80.1
74.8
77.9
70.9
45.3
78.9
43.0
76.6
Wan2.7
90.6
95.6
78.5
48.1
76.3
44.8
43.5
77.4
40.4
75.1
HappyHorse
95.8
74.4
74.5
53.3
73.1
31.7
49.9
76.0
48.8
73.2
Gemini-Omni
89.7
22.0
76.8
96.1
74.2
80.6
49.2
84.5
46.8
81.2
Seedance
69.1
47.8
83.0
80.4
74.9
60.1
54.0
84.4
48.5
82.8
LTX2
90.1
91.1
24.5
42.3
16.8
39.0
42.0
86.2
36.7
84.4
Figure 5: Metric orthogonality, single-modality editing, and error analysis. Left: Spearman correlation between subjective and objective metrics. Middle: video-only and audio-only performance, with axes normalized per metric. Right: per-model error counts across five failure categories.
Table 5: Human-LLM agreement of MLLM-as-Judge metrics. Agreement measures consistency between human annotations and automatic MLLM judgments, averaged over evaluated models. Values are percentages.
Metric
Editing Intent
Instruction Following
Fidelity Preserving
Realism
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
Agreement
89.9
90.0
89.9
91.3
92.9
89.7
88.8
87.9
90.1
91.0
93.0
89.0
Figure 6: Ablation of planner-side prompt enhancement and evaluator-driven retry refinement. Solid bars show the score without the module; hatched segments show the drop from the full AVE-Agent.
Table 6: Reference-free objective score routing in AVE-Compass. Each score is activated only for the instruction categories where it provides relevant evidence.
Metric Group
Score Name
Active Categories
Cross-Modal
lip_sync
speech
av_sync
audio-only, joint, speech
Video
video_aesthetic
video-only, joint, speech
subject_consistency
video-only, joint, speech
motion_smoothness
video-only, joint, speech
Audio
audio_aesthetic
audio-only, joint, speech
speech_quality
speech
Figure 7: Qualitative case analysis. Each row shows the source frame, edit instruction, and aligned edited frames from AVE-Agent, LTX2, Wan2.7, Seedance, Gemini-Omni, and HappyHorse. Red tags mark the dominant failure observed in each selected output, including low Fidelity Preserving under regeneration, over-realistic visual effects, non-response, physical-logic errors, spurious text/object insertion, temporal inconsistency, and poor Audio Fidelity Preserving.
Table 7: Difficulty-stratified analysis on AVE-Compass, each axis paired with its modality-matched metric. All values are on a 0–100 scale. Video Realism is normalized from the 1–5 rubric via (x−1)/4×100. Baselines degrade with difficulty, while our agent stays notably more robust under audio-source complexity and implicit cross-modal linkage.
Model
Object-Localization Hardness Video-IF
Object-Localization Hardness Video Realism
Audio-Source Complexity Audio Fidelity Preserving
Cross-Modal Linkage Audio-IF
easy
hard
easy
hard
simple
mod.
complex
explicit
implicit
AVE-Agent (Wan)
90.5
83.7
48.5
44.7
73.6
75.1
75.9
69.0
73.3
Wan2.7
89.3
76.3
47.3
42.7
60.3
46.3
36.0
61.3
56.0
HappyHorse
78.6
74.9
56.6
48.5
66.2
55.0
38.0
61.3
13.3
Gemini-Omni
73.1
74.2
54.4
48.1
100.0
96.4
91.4
10.5
13.3
Seedance
64.3
48.1
59.8
52.7
80.5
85.5
76.8
21.6
39.3
LTX2
75.6
72.2
37.3
43.0
52.0
35.8
36.4
65.4
72.7
Table 8: Single-modality editing (objective metrics). Top: video-only (n=16)—visual quality plus Audio Similarity (preservation of the untouched audio). Bottom: audio-only (n=5)—audio quality plus Video Similarity (preservation of the untouched video). Best in bold. ‡Our agent’s Audio Similarity is mildly depressed by a residual ∼1-frame muxing offset (frame-level alignment leaves an ∼8 ms residual); sample-aligned, the audio is essentially preserved.
Model
Video Aesthetic
Subject Consistency
Motion Smoothness
Audio Similarity
AVE-Agent (Wan)
0.463
0.983
0.985
0.969‡
Wan2.7
0.467
0.983
0.986
0.759
HappyHorse
0.454
0.986
0.989
0.999
Gemini-Omni
0.438
0.983
0.987
0.978
Seedance
0.442
0.980
0.985
0.833
LTX2
0.478
0.973
0.983
0.647
Table 9: Error analysis across five failure categories (error counts; lower is better). An edit is counted as an error in a category when it fails the corresponding checklist questions (per-case yes-rate <0.5) or, for audio-visual quality, when its Realism is below 3/5. Best (lowest) per row in bold; Gemini’s low Audio Fidelity Preserving count reflects non-response, not preservation quality.
Category
AVE-Agent (Wan)
Wan2.7
Happy Horse
Gemini Omni
Seedance
LTX2
1. Poor Video Fidelity Preserving (FP×video)
22
27
34
28
28
161
2. Poor Audio Fidelity Preserving (FP×audio)
37
97
93
5
34
106
3. Poor Video Instruction Following (IF×video)
20
32
39
40
88
44
4. Poor Audio Instruction Following (IF×audio)
48
69
78
144
133
56
5. Poor AV Quality (Realism <3/5)
48
58
50
31
31
39
Total
175
283
294
248
314
406
Table 10: MLLM-as-Judge agent-backbone ablation on the 34-case subset. Scores are on a 0–100 scale. All metrics are higher-is-better. Best results within each backbone pair are in bold.
Model
Editing Intent
Instruction Following
Fidelity Preserving
Realism
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
Overall
Video
Audio
AVE-Agent (Wan)
62.5+17.1
69.4+10.7
57.0+30.5
81.8+7.7
85.3+2.0
79.8+14.1
77.1+12.2
80.9+8.1
73.5+22.8
58.0+0.1
37.7+1.7
78.3-1.5
Wan
45.4
58.7
26.5
74.1
83.3
65.7
64.9
72.8
50.7
57.9
36.0
79.8
AVE-Agent (sd2)
49.0+16.2
44.2+10.5
52.0+25.1
70.0+26.3
69.6+19.1
68.5+31.9
70.0-12.6
64.7-17.5
77.2-6.1
64.8-8.4
50.2-10.1
80.3-5.7
Seedance
32.8
33.7
26.9
43.7
50.5
36.6
82.6
82.2
83.3
73.2
60.3
86.0
Table 11: Automated metric agent-backbone ablation on the 34-case subset. All metrics are higher-is-better. Best results within each backbone pair are in bold. †Speech Quality and Lip Sync are computed only on the two speech-category edits.
While instruction-based video editing has advanced rapidly, real-world videos contain tightly coupled audio and visual signals, and editing one modality often requires coordinated changes in the other. Existing benchmarks primarily evaluate visual transformations on silent clips or isolated audio editing, leaving complex audio-visual editing and cross-modal consistency underexplored. We introduce AVE-Compass, a comprehensive benchmark with 145 curated source videos, 196 audio-visually coupled ed
