Complementing Text Based Moderation with Multimodal Grounding: Detecting Fraudulent Inducement Behaviors on Short Video Platforms

What it claims

The paper adds a 7B multimodal LLM to an existing production OCR+LLM moderation pipeline. The goal is to catch fraudulent inducements in Chinese internet-finance short videos that leave no on-screen text, and to give reviewers time intervals instead of keyframes. The case for this rests on coverage: on the test set, about 50.4% of annotated events are non-textual (a QR code flash, a gesture toward a like button), spread over 458 of 725 videos (63%). A text-only pipeline cannot see these. The non-textual share is similar across classes (37.2% to 53.8%).

The authors set a hard constraint: detection recall must be at least as good as the text-only baseline before any gain in localization counts.

How it works

flowchart LR
  V[Short video] --> O[PaddleOCR PP-OCRv4]
  O --> T[Per-frame transcript]
  T --> L[LLM: class only, keyframe timing]
  T --> M[7B MLLM]
  V --> M
  M --> R[Intervals + classes]

PaddleOCR PP-OCRv4 transcribes on-screen text frame by frame, and the OCR stage is the same for all compared methods. The baseline LLM classifies that transcript alone. In the new design, a 7B MLLM receives the video plus the transcript. In one pass it emits a <think> reasoning trace, <answer> intervals [s,e], and <class> labels. The page does not name the base MLLM.

The model is post-trained with RL using four reward terms. The final design uses a recall-weighted F2 localization reward with class-balanced resampling.

Evidence

The dataset comes from about 20,000 screened Douyin and Kuaishou finance videos. It keeps 2,419 positive videos (1,694 train, 725 test), totalling 66.49 hours and 3,194 events, plus a small set of lifestyle videos as negatives. Test videos run 11.1 to 532.0 s (mean 106.3 s), with 1.36 events per video on average and at most 6. Test classes are skewed:

• simple interaction: 62.3%
• complex interaction: 22.7%
• page transition: 11.5%
• private conversion (QR codes, group IDs, the most fraud-sensitive class): 3.6%

Against the keyframe-based OCR+LLM baseline, OCR+MLLM reaches mIoU 26.52 versus 14.49, with 67.0% macro recall. The comparison with a large zero-shot MLLM is described along latency, visual token use and deployment complexity, but those figures are not in the fetched text.

The paper reports two reward-design findings it presents as transferable:

• Reward hacking. Auxiliary terms (center-distance penalties, event-count rewards) raised training reward while lowering real IoU.
• Coupling trade-off. Multiplicative coupling of IoU and class rewards learned localization but collapsed predictions to the dominant class. Additive coupling improved class balance but stalled localization. The authors attribute this to IoU giving no gradient when intervals do not overlap, combined with the 62% majority class.

They report a class-scaling "sweet spot" that only partially mitigates the trade-off.

Limits

The fetched text stops at the start of the reward-design section. Per-class results, reward formulas, latency and token numbers, and the zero-shot MLLM's scores are not available here. The authors state that a grounding MLLM alone degrades on mostly static frames, which is why OCR is kept.

An mIoU of 26.52 is still coarse. The paper frames its intervals as "approximate" regions for human review, not exact boundaries. The data is Chinese-platform finance content only. The page mentions no release of code, weights or the dataset, and gives no licence. It is a single-author v1 preprint dated 28 October 2026.