ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

What it claims

ALIVE is a first-frame-guided video editor built on LTX-2.5 with LoRA fine-tuning. It inserts an object so that the object responds to actions already in the source video: it gets picked up, worn, opened or cut. At inference the user supplies three things:

• the source video
• an edited first frame
• an instruction that only names the object (e.g. "Add the ceramic mug.")

The user does not supply masks, trajectories or an interaction description. An optional fine-tuned VLM predicts time-chunked interaction prompts from those same inputs. Figure 1 compares against Señorita qualitatively.

How it works

Data. The 35,800 training pairs come from four sources. Each pair is an object-absent source video X and an object-present target Y, and Y's first frame is the edit.

Qwen3.6-27B and GPT-5.6 filter out pairs with no visible interaction, poor visual quality, or bad object removal.

Prompt levels. Each pair is annotated at several levels:

• P0: no text.
• P1: object name only.
• P2: the principal interaction.
• P3: the sequence and outcome of the interaction.
• P4: per-chunk descriptions with frame intervals.

Qwen drafts the annotations and GPT reviews them. A single editor is trained on a mixture of P0–P4.

Conditioning.

• Clean source latents are channel-concatenated with noisy target latents, so the number of video tokens does not grow. The source half of the input projection is zero-initialised.
• The latent of the edited first frame is kept clean throughout denoising.
• The loss is native flow matching on tokens not conditioned on E₀.
• For P4, each chunk's text is encoded separately. Cross-attention for each video token is biased by log(temporal overlap with the chunk ÷ chunk text length). Zero overlap masks that chunk out, and the length normalisation stops longer descriptions from drawing more attention.
• All chunks are denoised jointly. Only video-to-text cross-attention is changed.

flowchart LR
  X[Source video latents] --> C[Channel concat]
  N[Noisy target latents] --> C
  E[Edited first frame, kept clean] --> D[Video DiT + LoRA]
  C --> D
  P1[Object-only instruction] --> Q{Guidance?}
  Q -->|P0–P3| G[Shared text context]
  Q -->|VLM predicts P4| T[Time-chunked contexts, overlap-weighted attention]
  G --> D
  T --> D

Evidence

• ALIVE-interaction benchmark (128 cases). In-domain held-out cases: 10 3D-rendered, 42 model-generated and 19 real-world. Out-of-domain: 31 from HOIGen-1M and 26 self-recorded (RealShot). Each case includes a reference video and P1–P4 annotations.
• General insertion benchmark (103 cases). 60 from ROSE and 43 from EffectErase.
• Scoring. Both benchmarks are scored by a single VLM-based protocol covering editing fidelity, source preservation and visual coherence.
• Headline results. As the summary states: +43.9% and +4.4% Overall over the strongest baseline without VLM guidance, and +0.95 points from predicted P4. The fetched text does not name which baseline is strongest.

Limits

• Truncated page. The fetched text stops at Section 4.3. The VLM's architecture and training details, the results tables, per-subset scores, the data-source ablation (§5.3.1) and the prompt-level comparison are not visible here.
• No adoption details. The page gives no code, weights, licence or compute figures. It links a project page only.
• Automated evaluation. All scores come from a VLM judge, not human ratings.
• Small subsets. Some benchmark slices are tiny, e.g. HOCap with 1 case and 3D-rendered with 10.
• Synthetic training data. Two-thirds of the training data is synthetic (rendered or generated), and the real and generated pairs depend on UnderEraser removal quality.
• Dependent backbone. The method depends on LTX-2.5, so porting it requires a DiT that accepts channel-concatenated source latents.