CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing
A new video-editing dataset and model let AI apply several edit instructions to one video at once, correctly and without mixing them up
Existing instruction-based video editing datasets mostly handle one edit at a time, like changing a background or removing an object, but real users often ask for several edits together. This paper releases CoinVE-200K, a 200K-pair dataset where each video comes with 2 to 5 combined edit instructions, plus CoinVE-Edit, a 22B-parameter model trained on it that keeps multiple instructions from interfering with each other, and CoinVE-Bench, a dedicated benchmark for this task.
METAL MEDIA explanatory visual
A new video-editing dataset and model let AI apply several edit instructions to one video at once, correctly and without mixing them up
- 01Starting from 1080p source videos up to 201 frames, the team combined six edit types across people, objects, and backgrounds (addition, removal, modification, stylization variants) into multi-instruction prompts, generated edited videos using a mask-guided pipeline, and strictly filtered results with Gemini 2.5 Pro to keep only 200K high-quality pairs.
- 02The final dataset averages 2.55 instructions per video (57.7% with 2, 31.7% with 3, 8.5% with 4, 2.1% with 5), spread fairly evenly across local addition (24.1%), background replacement (23.8%), object replacement (22.8%), local removal (18.5%), background stylization (6.4%), and object stylization (4.5%).
- 03CoinVE-Edit combines the Wan2.1-T2V-14B video generation backbone with the Qwen3VL-8B understanding model, predicting which spatial region each instruction should affect and disentangling attention so different edits don't leak into each other's regions.
- 04On both OpenVE-Bench (single instructions) and the new CoinVE-Bench (multiple instructions), CoinVE-Edit beat open-source baselines and even commercial systems Seedance 2.0 and Kling O3 on region-accuracy metrics (89.45 scope accuracy, 1.74 points above the best competitor), and outperformed simplified variants that just concatenated instructions or used a cruder region-injection method.
What they did
- Starting from 1080p source videos up to 201 frames, the team combined six edit types across people, objects, and backgrounds (addition, removal, modification, stylization variants) into multi-instruction prompts, generated edited videos using a mask-guided pipeline, and strictly filtered results with Gemini 2.5 Pro to keep only 200K high-quality pairs.
- The final dataset averages 2.55 instructions per video (57.7% with 2, 31.7% with 3, 8.5% with 4, 2.1% with 5), spread fairly evenly across local addition (24.1%), background replacement (23.8%), object replacement (22.8%), local removal (18.5%), background stylization (6.4%), and object stylization (4.5%).
- CoinVE-Edit combines the Wan2.1-T2V-14B video generation backbone with the Qwen3VL-8B understanding model, predicting which spatial region each instruction should affect and disentangling attention so different edits don't leak into each other's regions.
- On both OpenVE-Bench (single instructions) and the new CoinVE-Bench (multiple instructions), CoinVE-Edit beat open-source baselines and even commercial systems Seedance 2.0 and Kling O3 on region-accuracy metrics (89.45 scope accuracy, 1.74 points above the best competitor), and outperformed simplified variants that just concatenated instructions or used a cruder region-injection method.
| Dimension | Metric (Abbr.) | Range | Evaluator | Description |
|---|---|---|---|---|
| MLLM-Based (Gemini 3.6 Flash) | ||||
| Editing Accuracy (per Instruction) | Semantic Accuracy (SA) | [0,100] | MLLM + checklist | Correct execution of the intended edit semantics |
| Scope Accuracy (SPA) | [0,100] | MLLM + checklist | Correct edit localization without leakage or interference | |
| Editing Persistence (EP) | [0,100] | MLLM + checklist | Temporal consistency of the edit throughout the video | |
| Physical Naturalness | Appearance Naturalness (AN) | [0,100] | MLLM + checklist | Natural blending of lighting, shadows, textures, and style |
| Scale Consistency (SC) | [0,100] | MLLM + checklist | Plausibility of the edited object’s scale and perspective | |
| Motion Naturalness (MN) | [0,100] | MLLM + checklist | Plausibility of motion and physical interactions | |
| Semantic Preservation | Content Preservation (CP) | [0,100] | MLLM + checklist | Preservation of non-edited regions, objects, and structures |
| Specialized Evaluator-Based | ||||
| Video Quality | Aesthetic Quality (AQ) | [1,10] | Aesthetic Predictor v2.5 | Overall frame-level aesthetic appeal |
| Technical Quality (TQ) | (1, 100) | DOVER++ | Technical blur, noise, compression, flicker, and jitter | |
| Comprehensive Quality (CQ) | [1, 5] | VisualQuality-R1 | Overall frame-level visual quality | |
| Temporal Stability (TS) | [0, 1] | Optical-flow fields | Temporal motion smoothness |
| Model | Overall | Global Style | Background Change | Local Change | Local Remove | Local Add | Subtitle Edit |
|---|---|---|---|---|---|---|---|
| Runway | 3.51 | 3.72 | 2.62 | 4.18 | 4.16 | 2.78 | 3.62 |
| VACE | 1.55 | 1.49 | 1.55 | 2.07 | 1.46 | 1.26 | 1.48 |
| Ditto | 2.25 | 4.01 | 1.68 | 2.03 | 1.53 | 1.41 | 2.81 |
| OpenVE-Edit | 2.57 | 3.16 | 2.36 | 2.98 | 1.85 | 2.15 | 2.91 |
| VINO | 2.91 | 4.05 | 1.67 | 3.12 | 3.32 | 2.66 | 2.64 |
| OmniWeaving | 3.01 | 3.94 | 2.03 | 3.49 | 2.93 | 2.06 | 3.58 |
| KiwiEdit | 3.17 | 3.60 | 2.63 | 3.87 | 3.31 | 2.83 | 2.75 |
| SAMA | 3.33 | 3.87 | 2.64 | 3.91 | 3.31 | 2.63 | 3.63 |
| CoinVE-Edit | 3.41 | 3.61 | 3.11 | 3.84 | 3.81 | 2.53 | 3.53 |
| Model | Edit. Acc. | Phys. Natural. | Seman. Pres. | Video Quality | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SA | SPA | EP | AN | SC | MN | CP | AQ | TQ | CQ | TS | |
| Seedance 2.0 | 85.34 | 87.71 | 88.08 | 93.19 | 95.84 | 92.87 | 93.91 | 4.47 | 19.55 | 4.41 | 0.62 |
| Kling O3 | 86.91 | 80.93 | 89.06 | 92.55 | 90.30 | 93.91 | 84.51 | 4.49 | 18.36 | 4.37 | 0.61 |
| VACE | 3.98 | 17.15 | 6.50 | 26.69 | 13.82 | 15.21 | 87.83 | 4.05 | 17.59 | 4.11 | 0.62 |
| Ditto | 34.69 | 36.41 | 40.85 | 35.96 | 47.79 | 38.48 | 51.98 | 3.59 | 17.24 | 3.96 | 0.67 |
| VINO | 83.63 | 66.75 | 89.06 | 78.09 | 82.34 | 85.91 | 61.70 | 4.06 | 17.28 | 4.08 | 0.68 |
| OmniWeaving | 59.67 | 55.94 | 61.11 | 54.49 | 66.03 | 65.10 | 75.09 | 3.79 | 17.95 | 3.84 | 0.62 |
| KiwiEdit | 76.50 | 69.92 | 80.28 | 78.37 | 78.50 | 80.76 | 70.31 | 4.14 | 19.33 | 4.30 | 0.68 |
| SAMA | 75.58 | 73.35 | 79.63 | 83.43 | 83.88 | 88.14 | 90.08 | 3.61 | 18.08 | 4.19 | 0.72 |
| CoinVE-Edit | 87.97 | 89.45 | 89.60 | 91.85 | 91.17 | 95.30 | 90.83 | 4.13 | 19.57 | 4.31 | 0.72 |
| Variant | Components | Edit. Acc. | Phys. Natural. | Seman. Pres. | |||||
|---|---|---|---|---|---|---|---|---|---|
| Mask | Q-Blending | SA | SPA | EP | AN | SC | MN | CP | |
| Instr.-Concat. | × | × | 82.61 | 84.43 | 89.17 | 86.52 | 90.02 | 92.17 | 89.30 |
| Q-Bias | ✓ | × | 83.35 | 85.49 | 88.19 | 87.08 | 89.64 | 91.95 | 90.13 |
| CoinVE-Edit | ✓ | ✓ | 87.97 | 89.45 | 89.60 | 91.85 | 91.17 | 95.30 | 90.83 |
Why it matters
Real-world editing requests usually bundle multiple changes rather than a single simple edit, so this dataset and model give researchers a concrete way to train and measure whether AI systems can apply each request to the right region without disturbing the rest of the video. Because the benchmark shows gaps even in leading commercial video models, it also gives the industry a target for improving compositional editing quality.
Terms in this paper
- MLLM (multimodal large language model) · a large language model that can understand both images/video and text together
- DiT (Diffusion Transformer) · a video/image generation model that uses a transformer architecture inside a diffusion process
- attention · the mechanism a model uses to decide which parts of the input to focus on
- SAM2/SAM3 · segmentation models that automatically find the region (mask) of a target object in images or video
- Q-Blending cross-attention · CoinVE-Edit's attention method that softly injects each instruction's region information to avoid interference between edits
Figures we cannot republish
- Figure 1: Demonstration of compositional instruction-guided video editing cases from our proposed CoinVE-200K.
- Figure 2: Data construction pipeline of CoinVE-200K.
- Figure 3: Data statistics of CoinVE-200K.
- Figure 4: An overview of the proposed CoinVE-Edit framework.
- Figure 5: Quality comparison of single-instruction video editing on OpenVE-Bench.
- Figure 6: Quality comparison of compositional-instruction video editing on CoinVE-Bench.
- Figure 7: Visualization of predicted editing mask of each instruction for two editing examples.
Original abstract (English)
The quality and diversity of instruction-based video editing datasets are steadily improving, yet existing datasets mainly focus on single editing operations and fall short in supporting compositional instruction-guided video editing. In particular, multiple editing intents must be jointly understood and faithfully executed within the same video. To address this issue, we introduce CoinVE-200K, a large-scale, high-quality dataset for Compositional Instruction-Guided Video Editing. CoinVE-200K contains 1080p video-editing pairs of up to 201 frames, covering diverse compositional scenarios where each sample involves 2 to 5 atomic editing operations. The instructions target humans, objects, and backgrounds, and cover edit types such as addition, removal, modification, and stylization. All samples are built through a carefully designed generation and filtering pipeline to ensure instruction faithfulness, visual quality, temporal consistency, and compositional diversity. We also introduce CoinVE-Bench, a benchmark for compositional-instruction video editing across diverse subjects, operation types, and instruction complexities. Furthermore, we present CoinVE-Edit, a 22B compositional video editing model built upon Wan2.1-T2V-14B and Qwen3-VL-8B-Instruct. CoinVE-Edit disentangles region-aware attention for different editing instructions, enabling precise multi-region editing while preserving irrelevant content and temporal coherence. Experiments on CoinVE-Bench show that CoinVE-Edit achieves strong performance in instruction following, compositional editing accuracy, visual quality, and temporal consistency.
Read on arXivLatest papers
- SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the time
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papers
- PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad action
- EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise it
- EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses
- Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversight
- PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappears
- Automated Summarization of Financial News Using Large Language Models and Retrieval-Augmented Generation: An Early Empirical Study (Fall 2023)Testing AI summaries of stock news, the simple approach beat the trendy retrieval-based one