CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing

arXiv:2608.175662026-08-17

新数据集和模型让AI能在一个视频里同时准确执行多条编辑指令而互不干扰

现有的指令视频编辑数据集大多只处理单一编辑操作,比如换背景或去掉一个物体,但用户实际常常一次性提出多个编辑需求。这篇论文发布了CoinVE-200K,一个每个视频配有2到5条组合编辑指令、共20万对的数据集,以及基于此训练的220亿参数模型CoinVE-Edit,能让多条指令各自作用在正确区域而不互相干扰,同时还配套发布了专用评测基准CoinVE-Bench。

METAL MEDIA 解读图

新数据集和模型让AI能在一个视频里同时准确执行多条编辑指令而互不干扰

01从最长201帧、1080p分辨率的原始视频出发,团队围绕人物、物体、背景组合了添加、删除、修改、风格化等六种编辑类型生成多指令提示词,用基于分割掩码的方式合成编辑后视频,再用Gemini 2.5 Pro严格筛选,最终保留20万条高质量视频编辑对。
02最终数据集平均每个视频含2.55条指令(含2条的占57.7%、3条31.7%、4条8.5%、5条2.1%),编辑类型分布较均衡:局部添加24.1%、背景替换23.8%、物体替换22.8%、局部删除18.5%、背景风格化6.4%、物体风格化4.5%。
03CoinVE-Edit结合了Wan2.1-T2V-14B视频生成骨干和Qwen3VL-8B理解模型,通过预测每条指令对应的视频空间区域,并将不同指令的注意力机制解耦,使编辑只作用于对应区域,避免相互干扰。
04在单指令基准OpenVE-Bench和新提出的多指令基准CoinVE-Bench上,CoinVE-Edit不仅超过开源模型,在区域编辑准确率等指标上也优于商业模型Seedance 2.0和Kling O3(区域准确率89.45,领先最佳竞品1.74),同时优于把指令直接拼接或采用简单区域注入方式的简化版本。

这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

从最长201帧、1080p分辨率的原始视频出发,团队围绕人物、物体、背景组合了添加、删除、修改、风格化等六种编辑类型生成多指令提示词,用基于分割掩码的方式合成编辑后视频,再用Gemini 2.5 Pro严格筛选,最终保留20万条高质量视频编辑对。
最终数据集平均每个视频含2.55条指令(含2条的占57.7%、3条31.7%、4条8.5%、5条2.1%),编辑类型分布较均衡:局部添加24.1%、背景替换23.8%、物体替换22.8%、局部删除18.5%、背景风格化6.4%、物体风格化4.5%。
CoinVE-Edit结合了Wan2.1-T2V-14B视频生成骨干和Qwen3VL-8B理解模型,通过预测每条指令对应的视频空间区域,并将不同指令的注意力机制解耦,使编辑只作用于对应区域,避免相互干扰。
在单指令基准OpenVE-Bench和新提出的多指令基准CoinVE-Bench上,CoinVE-Edit不仅超过开源模型,在区域编辑准确率等指标上也优于商业模型Seedance 2.0和Kling O3(区域准确率89.45,领先最佳竞品1.74),同时优于把指令直接拼接或采用简单区域注入方式的简化版本。

Table 1: Evaluation matrix of CoinVE-Bench. The proposed metrics are grouped by evaluator type into MLLM-based checklist evaluation and specialized evaluator-based evaluation with task-specific models, covering four evaluation dimensions and eleven fine-grained metrics.
Dimension	Metric (Abbr.)	Range	Evaluator	Description
MLLM-Based (Gemini 3.6 Flash)
Editing Accuracy (per Instruction)	Semantic Accuracy (SA)	[0,100]	MLLM + checklist	Correct execution of the intended edit semantics
Scope Accuracy (SPA)	[0,100]	MLLM + checklist	Correct edit localization without leakage or interference
Editing Persistence (EP)	[0,100]	MLLM + checklist	Temporal consistency of the edit throughout the video
Physical Naturalness	Appearance Naturalness (AN)	[0,100]	MLLM + checklist	Natural blending of lighting, shadows, textures, and style
Scale Consistency (SC)	[0,100]	MLLM + checklist	Plausibility of the edited object’s scale and perspective
Motion Naturalness (MN)	[0,100]	MLLM + checklist	Plausibility of motion and physical interactions
Semantic Preservation	Content Preservation (CP)	[0,100]	MLLM + checklist	Preservation of non-edited regions, objects, and structures
Specialized Evaluator-Based
Video Quality	Aesthetic Quality (AQ)	[1,10]	Aesthetic Predictor v2.5	Overall frame-level aesthetic appeal
Technical Quality (TQ)	(1, 100)	DOVER++	Technical blur, noise, compression, flicker, and jitter
Comprehensive Quality (CQ)	[1, 5]	VisualQuality-R1	Overall frame-level visual quality
Temporal Stability (TS)	[0, 1]	Optical-flow fields	Temporal motion smoothness

Table 2: Single-instruction video editing comparison on OpenVE-Bench evaluated by Gemini 2.5 Pro.
Model	Overall	Global Style	Background Change	Local Change	Local Remove	Local Add	Subtitle Edit
Runway	3.51	3.72	2.62	4.18	4.16	2.78	3.62
VACE	1.55	1.49	1.55	2.07	1.46	1.26	1.48
Ditto	2.25	4.01	1.68	2.03	1.53	1.41	2.81
OpenVE-Edit	2.57	3.16	2.36	2.98	1.85	2.15	2.91
VINO	2.91	4.05	1.67	3.12	3.32	2.66	2.64
OmniWeaving	3.01	3.94	2.03	3.49	2.93	2.06	3.58
KiwiEdit	3.17	3.60	2.63	3.87	3.31	2.83	2.75
SAMA	3.33	3.87	2.64	3.91	3.31	2.63	3.63
CoinVE-Edit	3.41	3.61	3.11	3.84	3.81	2.53	3.53

Table 3: Compositional-Instruction Video Editing Comparisons on CoinVE-Bench.
Model	Edit. Acc.	Phys. Natural.	Seman. Pres.	Video Quality
SA	SPA	EP	AN	SC	MN	CP	AQ	TQ	CQ	TS
Seedance 2.0	85.34	87.71	88.08	93.19	95.84	92.87	93.91	4.47	19.55	4.41	0.62
Kling O3	86.91	80.93	89.06	92.55	90.30	93.91	84.51	4.49	18.36	4.37	0.61
VACE	3.98	17.15	6.50	26.69	13.82	15.21	87.83	4.05	17.59	4.11	0.62
Ditto	34.69	36.41	40.85	35.96	47.79	38.48	51.98	3.59	17.24	3.96	0.67
VINO	83.63	66.75	89.06	78.09	82.34	85.91	61.70	4.06	17.28	4.08	0.68
OmniWeaving	59.67	55.94	61.11	54.49	66.03	65.10	75.09	3.79	17.95	3.84	0.62
KiwiEdit	76.50	69.92	80.28	78.37	78.50	80.76	70.31	4.14	19.33	4.30	0.68
SAMA	75.58	73.35	79.63	83.43	83.88	88.14	90.08	3.61	18.08	4.19	0.72
CoinVE-Edit	87.97	89.45	89.60	91.85	91.17	95.30	90.83	4.13	19.57	4.31	0.72

Table 4: Performance comparison among different variants of CoinVE-Edit on CoinVE-Bench.
Variant	Components	Edit. Acc.	Phys. Natural.	Seman. Pres.
Mask	Q-Blending	SA	SPA	EP	AN	SC	MN	CP
Instr.-Concat.	×	×	82.61	84.43	89.17	86.52	90.02	92.17	89.30
Q-Bias	✓	×	83.35	85.49	88.19	87.08	89.64	91.95	90.13
CoinVE-Edit	✓	✓	87.97	89.45	89.60	91.85	91.17	95.30	90.83

为什么重要

现实中的视频编辑需求往往是多个改动叠加而非单一操作,这套数据集和模型为训练与检验AI能否把每条指令精确落到对应区域、同时保持其余内容不变提供了具体基础。评测结果显示即便是领先的商业视频模型也存在明显不足,这也为整个行业提升组合编辑质量提供了可参照的目标。

本文术语

MLLM(多模态大语言模型) · 能够同时理解图像/视频和文本的大型语言模型
DiT(扩散Transformer) · 在扩散生成过程中使用Transformer结构的视频/图像生成模型
注意力机制(attention) · 模型用来决定关注输入哪些部分的计算方式
SAM2/SAM3 · 能在图像或视频中自动找出目标对象所在区域(掩码)的分割模型
Q-Blending交叉注意力 · CoinVE-Edit中用于柔和地注入各指令对应区域信息、减少指令间干扰的注意力方法

无法转载的图表

Figure 1: Demonstration of compositional instruction-guided video editing cases from our proposed CoinVE-200K.
Figure 2: Data construction pipeline of CoinVE-200K.
Figure 3: Data statistics of CoinVE-200K.
Figure 4: An overview of the proposed CoinVE-Edit framework.
Figure 5: Quality comparison of single-instruction video editing on OpenVE-Bench.
Figure 6: Quality comparison of compositional-instruction video editing on CoinVE-Bench.
Figure 7: Visualization of predicted editing mask of each instruction for two editing examples.

在原文中查看图表 →

论文原文摘要（英文）

The quality and diversity of instruction-based video editing datasets are steadily improving, yet existing datasets mainly focus on single editing operations and fall short in supporting compositional instruction-guided video editing. In particular, multiple editing intents must be jointly understood and faithfully executed within the same video. To address this issue, we introduce CoinVE-200K, a large-scale, high-quality dataset for Compositional Instruction-Guided Video Editing. CoinVE-200K contains 1080p video-editing pairs of up to 201 frames, covering diverse compositional scenarios where each sample involves 2 to 5 atomic editing operations. The instructions target humans, objects, and backgrounds, and cover edit types such as addition, removal, modification, and stylization. All samples are built through a carefully designed generation and filtering pipeline to ensure instruction faithfulness, visual quality, temporal consistency, and compositional diversity. We also introduce CoinVE-Bench, a benchmark for compositional-instruction video editing across diverse subjects, operation types, and instruction complexities. Furthermore, we present CoinVE-Edit, a 22B compositional video editing model built upon Wan2.1-T2V-14B and Qwen3-VL-8B-Instruct. CoinVE-Edit disentangles region-aware attention for different editing instructions, enabling precise multi-region editing while preserving irrelevant content and temporal coherence. Experiments on CoinVE-Bench show that CoinVE-Edit achieves strong performance in instruction following, compositional editing accuracy, visual quality, and temporal consistency.

作者 · Fuchen Long

在 arXiv 阅读

CoinVE-200K: A Large-Scale High-Quality Dataset for Compositional Instruction-Guided Video Editing

新数据集和模型让AI能在一个视频里同时准确执行多条编辑指令而互不干扰

他们做了什么

为什么重要

本文术语

无法转载的图表

最新论文

METAL MEDIA 最新报道