Figure 1: Illustrative multi-turn policy comparison. A click-supervised policy without visual-consistency supervision (left) proposes a slate containing a missing-source edit and an already-satisfied target; the user selects the executable lighting edit. On the updated image, the full framework (right) produces executable, image-consistent, and non-redundant follow-up edits.
Figure 2: Three-stage training framework. Stage 1 constructs validated SFT slates from real contexts. Stage 2 trains a click reward model and optimizes five non-grounding rewards with GRPO. Stage 3 derives an image-first source–target grounding reward and supplies it as a sixth signal to a separate GRPO run, initialized from SFT and otherwise using the Stage 2 pipeline.
Table 2: Cumulative ablation of RL components. All rows start from the SFT policy.
RL configuration
GSB vs base ↑
Ground. ↓
Redund. ↓
SFT initialization
+332
3.0
17.2
+ core RL rewards
+374
4.4
28.1
+ max-pair diversity
+383
4.0
19.1
+ norm./dynamic weights
+405
3.7
11.9
+ grounding reward
+𝟒𝟒𝟔
0.9
8.8
Figure 3: SFT data-construction pipeline (general track). Pale blocks are the data surviving each stage; grey ribbons show removed records and their filtering reasons.
Table 4: Verifier evaluation on the visual-consistency calibration set.
Visual inconsistency recall ↑
False
Verifier
Missing src.
Already sat.
Union
rejection ↓
Single-pass VLM
61.6%
43.4%
47.5%
22.2%
Source–target
92.9%
74.5%
78.7%
0.6%
Table 5: Stage-wise visual inconsistency by error type. Union denotes the rate of suggestions with either error.
Policy
Missing source ↓
Already-satisfied target ↓
Visual inconsistency (union) ↓
SFT (Stage 1)
0.74%
2.26%
3.0%
SFT + RL (Stage 2)
0.70%
3.00%
3.7%
Full framework (Stage 3)
0.42%
0.48%
0.9%
Table 6: Preference-data funnel. Counts in the first three rows are impression-level unless marked as suggestion-level.
Stage
Filtering operation
Output
Source
Three displayed suggestions and a non-empty click
663,926 requests
Record filtering
Remove click not in display (1,861), invalid intent (45,022), invalid clicked type/sub-tag (120,344), fewer than two usable continuation suggestions (15,344), and repeated sessions (249,062)
232,293 impressions
Image transfer
Retain records with a successfully transferred image
232,281 impressions
Pair eligibility
Remove unusable query (944), query-copy click (66), missing final intent (30), and no usable suggestion above the click (102,491)
128,750 impressions
Suggestion pruning
Drop generic refresh-like (39,829), non-continuation (45,420), and below-click options (204,173); these are suggestion-level counts
173,071 pairs
Request split
Group by request ID
164,401 train / 8,670 validation pairs
Table 7: Per-candidate dimensions emitted by one verifier call.
Dimension
Judged from
Used for
missing source
source existence bits with evidence
RL grounding g^
already satisfied
target vs. current state
RL grounding g^
plausibility
fit to image and scene
monitoring only
effective change
non-trivial visible change
monitoring only
content density
global appearance vs. object/region edit
length budget class
evidence, states
pointing phrases
audits and error analysis
Table 8: Source and target semantics by edit type.
Operation
Required source
Target check
Add
anchor/context entity
requested addition not satisfied
Remove
object to remove
object not already absent
Modify
object and current state
new state not already satisfied
Relocate
object and reference region
requested relation not satisfied
Global style
image
style change checkable or uncertain
Table 9: Diversity signal and aggregation on the semantic-redundancy probe.
Variant
Adapt. λ
Redund. ↓
Near-dup ↓
SFT
×
14.8%
1.4%
Jaccard
×
28.0%
2.2%
Embed., mean
×
30.0%
2.2%
Embed., max-pair
×
20.8%
0.8%
Embed., max-pair
✓
9.6%
0.4%
Table 10: RM-only genericization stress test. Cross-image reuse and within-list redundancy are measured at the final checkpoint.
Figure 4: Source–target visual consistency checks. A suggestion is invalid when it presupposes an absent source or requests a visually checkable target state that already holds.
Conversational assistants increasingly recommend follow-up edits to help users continue a task. Existing systems primarily target text-only interactions, leaving image-creation conversations underexplored. In image-creation tasks, useful follow-up edit suggestions must reflect user preferences, offer diverse directions, and remain executable on the current image. We collected 100,000 real multi-turn image-creation conversation samples from Qwen App and found that 80.1% are image-dependent, underscoring the need for multimodal recommendation. We address this setting with a three-stage framework. In Stage 1, we use real online data to build a human-reviewed table of appropriate follow-up editing intents, then create SFT targets and fine-tune a multimodal policy. In Stage 2, to align rule-guided SFT suggestions with actual user choices, we use user click feedback to optimize the policy through multi-objective reinforcement learning. In Stage 3, to reduce visual inconsistencies between suggested edits and the current image, we introduce a visual verifier as additional training supervision. Extensive experiments demonstrate that our framework significantly outperforms baselines on both automatic and human evaluations. In a live user-randomized A/B test with millions of users, our final framework reduces visual inconsistency from 3.7% to 0.9%. Furthermore, it significantly improves recommendation CTR by 32.70%, image take-away rate by 16.32%, and average conversation turns per user by 39.90% (all p<0.05).
