90 results for “Mage”

JADE (Joint Action DEnoiser) is a diffusion-based vision-language-action architecture in which the VLM reads the current noisy action latents together with images and instructions at every denoising step, and guides the action expert through gated cross-attention. The VLM is also trained to recover clean action latents, and an adaptive step-size schedule cuts sampling steps. On RoboDojo it averages 12.67% success, the highest among π0.5-based methods, and it adds 20.8 points over π0.5 on four real bimanual tasks.

arxiv.orgpaperAdded 3 Nov 2026, published 31 Oct 2026🧠 AI

The paper audits six situational-safety benchmarks for multimodal LLMs and proposes the Benchmark Card, seven measured fields that show whether a SAFE/UNSAFE label score comes from the scene, the request or the benchmark files. MSSBench chat leaks its label through image size, which alone reaches 82.3% accuracy. The Spot-Stop decomposition splits ranking-versus-labelling gaps across fourteen open MLLMs, and label-free recentering improves ranking on VSCBench for 13 of 14 models.

arxiv.orgpaperAdded 3 Nov 2026, published 31 Oct 2026🧠 AI

RADIOlogy is a training-free, single-pass zero-shot anomaly detection method that uses multi-head attention pooling in vision-language models. It scores anomalies by contrasting average similarity between vision features and sets of normal and abnormal prompts, with no windowed execution. RADIO agglomerative features drive a dense refinement pipeline for pixel-level segmentation, and the method is applied to medical images.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

PhysCount is a benchmark of 546 real-world images and 4,595 human-written questions in 651 target-specific groups. It tests whether multimodal LLMs can separate visible appearances (direct view, reflections, glass) from the physical entities behind them. Across 40 MLLMs, counting appearances is much easier than linking them to entities, and the best model answers every question correctly in only 13.36% of groups.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

inkGround is a benchmark for stroke trajectory recovery from full-page, camera-captured handwriting, with 537 page images (about 18.7k words, 163k strokes) and pseudo-label references filtered by OCR agreement and human raters. Models get a page image and a target word, then output ordered stroke points. Seven VLMs were scored with DTW, Chamfer distance, readability and style similarity. Closed-source models, notably Gemini-3.1-Pro, did best, and the ranking held on IAM-OnDB.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

Beyond the Event Horizon of Keccak extends a 2011 observation that a path in the Keccak sponge from a zero-capacity state back to one yields infinite second-preimage families. Three theorems cover loops from any capacity value x, k-block loops and cycles, and Joux-style multicollisions with search cost independent of k. Verified examples run on Keccak-f[200] and Keccak-f[1600] with c=32; it also proposes bridge-based protocols.

eprint.iacr.orgpaperPublished and added 28 Oct 2026🛡️ Security

Know, then Draw is a learning paradigm for visual generation that uses autoregressive pretraining to acquire semantic knowledge and diffusion post-training for high-fidelity rendering. Controlled comparisons find autoregression the stronger pretraining objective, and semantic quality matters more than AR-only generation quality. This motivates a purely semantic tokenizer trained with language supervision only. A 3B text-to-image model trained in 6,600 H100 GPU-hours scores 86 on GenEval and 87 on DPG-Bench.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

This paper presents an annotated egocentric tennis dataset (egocentric video synced with five allocentric views, global positions, contact times, stroke types) and evaluates image encoders, video encoders and vision-language models on seven prediction tasks. Best models reach 0.14 m player and 0.54 m ball localization error and 0.14 s contact-time error. Depth input helps 3D ball localization, and longer histories do not help forecasting. A conditional diffusion model generates whole-body tennis motion in real time.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

This paper studies how multimodal LLMs refuse questions that an image cannot answer, using the MoHoBench benchmark on three MLLMs. It varies the meaning and placement of refusal instructions, compares supervised and preference-based post-training, and intervenes on a refusal-related activation direction. Instructions change refusal at the sample level in ways aggregate rates hide. Post-training raises refusal most but can mix refusal with attempted answers. Activation intervention gives smaller, transferable changes.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

Falko Strenzke's ePrint 2026/2635 presents a multi-target second-preimage attack on message hashing in HSS/LMS and XMSS with SHA-256 and SHA-256/192. It herds chaining states from many collected signatures into one state, defeating the key and leaf-index prefix. With 2^64 signatures, SHA-256 cost falls from 2^256 to about 2^166 (2^156 at maximal message length). SHAKE256 parameter sets are unaffected.

eprint.iacr.orgpaperAdded 26 Oct 2026, published 25 Oct 2026🛡️ Security

OWLS (Open-world Language-conditioned Speed Fields) plans directly over a vision-language model's dense feature map. A text prompt describing traversable terrain scores every pixel of one camera image. The scores are projected onto the ground, adapted with two morphological operations, and used as the speed field of a minimum-time planner. It needs no training, terrain labels, map or extra sensors. It was tested on a physical rover, on simulated Mars and Moon rovers over 200 procedural terrains, and on real planetary imagery.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

DeepFaune-UK is a fine-tune of the open DeepFaune camera-trap classifier on 28 UK classes, paired with MegaDetector v6. On 20,212 MammalWeb images it reached 88.8% strict accuracy against 56.5% (DeepFaune v1.5.0) and 59.0% (SpeciesNet). Re-applying the recipe to the DINOv3-based classifier gave 90.6%. Fine-tuning on about 150 crops per class matched full-data out-of-distribution accuracy, and a DINOv2 run took about 22 minutes on under 4 GB of GPU memory.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

UAV-SwarmAgent is a hierarchical multi-agent system where an operator gives one natural-language sentence and a ground-station agent writes and runs code that produces per-UAV routes. A vision-language model grounds targets in an aerial map, and worker agents verify incidents with a compact onboard model. In simulation it located the incident in 244 of 255 missions (95.7%), and flight time followed an Amdahl-type law with speedup bounded at 1.52x for one to five UAVs.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

Decode-time provenance binding is a defense for Model Context Protocol agent pipelines against cross-modal trust laundering, where instructions hidden in an email-signature image via LSB, DCT or metadata steganography get decoded by a tool. It combines HMAC-authenticated trust labels, a provenance ledger and context-taint tracking to block critical tool calls. On a real MCP testbed with 24 adaptive payloads, attack success fell from 100% undefended (62.5% for keyword and regex filters) to 0%.

arxiv.orgpaperAdded 26 Oct 2026, published 22 Oct 2026🛡️ Security

F2G-Pose is a direct, single-pass framework that predicts category-level object pose, metric size and camera-frame completed shape from one RGB-D image. It lifts frozen RADIO foundation features onto partial point clouds, tokenizes local geometry with DGCNN, and fuses it with a Mixture-of-Experts Transformer. It trains only on synthetic SOPE data, leads on SOPE and ROPE at 28 FPS, and is tested on HouseCat6D, HANDAL and real manipulation.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

TRIAD is a benchmark for omni-modal ambiguity resolution. It has 327 hand-written text-image-audio queries in 86 scene groups across 18 ambiguity categories, and removing any one modality makes the answer non-unique. Across 14 omni-modal systems, the best model trails humans by over 20 points in group accuracy. Without answer options, model group accuracy falls to single digits, and leave-one-out tests show audio cues are often underused.

arxiv.orgpaperAdded 23 Oct 2026, published 22 Oct 2026🧠 AI

MPIDetector is a detection framework for multi-modal prompt injection that uses few-shot in-context learning so an LLM can reason about the semantic intent of text and image inputs, without fine-tuning. The authors built a benchmark of eight textual and seven visual attack categories plus benign sets. They report up to 90.91% accuracy on text, 99.28% on images, and up to 99.80% detection on 500 obfuscated WebInject web-agent injections, beating baselines.

arxiv.orgpaperAdded 24 Oct 2026, published 21 Oct 2026🛡️ Security

AnomalyAgent is a post-training fusion framework for visual anomaly detection that ensembles pretrained detectors. It aligns scores and anomaly maps, summarizes disagreement, and treats fusion as a choice among score actions and map actions. A constrained vision-language model inspects high-conflict samples and picks the actions. It beats equal-weight fusion and gets the best mean image AUROC and pixel AP among compared fusion methods on MVTec AD and VisA.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

Code2Video-Compass is a benchmark of 2,104 tasks for code-based video generation, where multimodal LLMs write executable programs that produce complete videos from text (T2V), image (I2V) or video (V2V) inputs. Tasks span 10 categories and 24 subtypes. Scoring uses 12 objective metrics and five MLLM-judge metrics. Across 15 proprietary and open-weight models from nine families, performance falls as construction, dynamic animation and state-dependency complexity rise.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

This paper presents a multimodal engineering GraphRAG framework that links post-wind-event damage evidence from images and reconnaissance reports to building-condition states, recovery actions and downtime. Evidence is stored as a provenance-preserving, component-level representation, and graph-aligned retrieval uses compatibility, contradiction, evidence-precedence and abstention checks. Demonstrated on tornado-affected low-rise homes, it beats BaselineRAG and Structured Baseline RAG, and ablations test each component.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

OpenVLHarness is a multimodal reasoning harness for MLLMs built on a capability layer (task-level tool interfaces plus evidence rendering) and persistent multimodal memory that stores images, structured data and web references as addressable artifacts. A coding interface runs generated programs over stored data. Across 22 benchmarks and eight backbones it adds 9.7-12.6 points over no-tool baselines, and GPT-6 Luna at medium effort beats Codex and Visual Sketchpad at maximum effort using at most 37% of their tokens.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

FedRAPP is a personalized aggregation framework for Source-Free Federated Test-Time Prompt Tuning (SF-FTTPT), where clients adapt CLIP soft prompts on unlabeled, non-IID test streams without source data. It combines reliability-aware client assessment, a server-side exponential moving-average prompt memory and similarity-guided peer aggregation. Across nine CLIP benchmark datasets and ImageNet variants it beats standard federated aggregation baselines while keeping prompt-only efficiency.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

LILIE (Language-Informed and Lossless Image Encoder) is an invertible image transform built from a normalizing flow transformer, trained jointly with a vision-language model via next token prediction so captions can be recovered from noised features. The encoder supplies a language-informed loss for text-to-image generation. JiT models trained with it cut gFD from 66.84 to 53.01 against a pixel-only baseline on the GPIC dataset. Code, encoders and models are released.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

This paper presents a multimodal retrieval-augmented generation framework for Medical Visual Question Answering. A Gated Multimodal Unit learns a joint query representation from the medical image and the clinical question, so evidence retrieval uses both modalities while the Med-LVLM stays frozen. On benchmark datasets it consistently beats a reference Med-LVLM and stays competitive with state-of-the-art methods. The page gives no dataset names or numbers.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

P^3-WAM is a unified world action model for egocentric dexterous manipulation with three modes: Perceive (reconstruct articulated hand motion and camera trajectories from video), Predict (generate visual futures under prescribed actions) and Plan (jointly generate action trajectories and their anticipated visual outcomes from an initial image, a language instruction and an optional visual goal). It adds a compact structured 3D action representation to a pretrained video diffusion transformer through a mixture-of-transformers design with shared base weights, modality-specific adaptation and bidirectional visual-action attention. Randomized modality visibility lets video and 3D actions act as either context or target. Unified training beats single-mode training and outperforms specialized baselines on 3D hand reconstruction and action-conditioned video prediction across four datasets, and gives better instruction adherence than video-generation baselines.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

GhostFaceMPC is presented as the first end-to-end face recognition system that runs both the deep CNN feature extractor and the matching step under secure multi-party computation. Raw images and biometric embeddings are never exposed at any stage, from feature extraction through storage to matching. The authors contribute an MPC-friendly quantization of the GhostFaceNet model and circuit optimizations that make this practical, and they release code on GitHub. It is relevant as an example of making CNN inference feasible under cryptographic constraints.

eprint.iacr.orgpaperPublished and added 15 Oct 2026🧠 AI

GhostFaceMPC is presented as the first end-to-end face recognition system that runs a deep CNN feature extractor (GhostFaceNet) and the matching step under secure multi-party computation. Raw images and biometric embeddings are never exposed during extraction, storage or matching. The authors contribute an MPC-friendly quantization of GhostFaceNet and circuit optimizations that make the approach practical, and they release code at crypto-unict/GhostFaceMPC.

eprint.iacr.orgpaperPublished and added 15 Oct 2026🛡️ Security

MiniDog is a framework for training a text-to-image flow-matching diffusion transformer from scratch on four RTX 3090 GPUs, with 55 minutes of pretraining and 15 minutes of fine-tuning. It pretrains on 26k ImageNet dog images that were captioned, then fine-tunes on 2k FLUX.1-schnell generated images. Image and text features are precomputed and cached. The model is a 22M-parameter transformer with the E2E-INVAE semantic latent space, a small Qwen text encoder and in-context conditioning, and training is accelerated with iREPA and an EUPE perception encoder. The repo evaluates with FID, HPSv2, PickScore and VAE reconstruction metrics. It lowers the barrier to hands-on diffusion training research.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Proposes mmVB, a lightweight plug-and-play module (1.6M trainable parameters) that assigns query-conditioned per-image visual token budgets using cross-image context for multimodal LLMs, without retraining the base model or accessing attention matrices (so it is compatible with FlashAttention). It cuts visual input tokens 9x, giving 5x faster time-to-first-token and 7x fewer prefill FLOPs while keeping 94.4% of full-resolution accuracy across eight multi-image reasoning benchmarks. It matters because existing token pruning, merging and compression methods target single-image redundancy and either need attention maps or retraining.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

PixelMoE is a pixel-space diffusion framework that scales capacity sparsely using standard LLM-style Mixture-of-Experts: token-choice routing, fine-grained routed experts, a shared expert and bias-based load balancing, with no diffusion-specific routing or auxiliary routing losses. It adds Pixel-Space Hierarchical Flow Matching, which varies spatial resolution along the denoising trajectory from global structure to fine detail. At matched active parameters it beats dense counterparts, and reaches FID 1.47 at 256x256 and 1.53 at 512x512 on class-conditional ImageNet-1K without representation alignment or perceptual losses. Code and models are promised but not yet released.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

OmniCompass is a manually annotated benchmark for video grounding and understanding with two tracks: OmniCompass-VL (820 videos, 2,460 instances; vision-language models) and OmniCompass-AV (240 videos, 720 instances; joint audio-visual evidence for omni-modal models). Each track covers temporal grounding, spatial grounding and QA across short/medium/long videos, and each instance is run in paired text-only and text+image target specifications. Evaluating 50 MLLMs shows open-source models lag proprietary ones, especially on audio-visual understanding, that precise temporal/spatial grounding remains hard, and that predictions are inconsistent across the paired specifications.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

EmoGenie is a benchmark for emotions expressed and evoked by text-to-image outputs. It has 6,400 images from eight T2I models using 800 emotion-targeted prompts. It includes EmoGenie-QA (90,000 closed-set and 1,057 open-ended questions) and EmoGenie-Assess (13,175 mean opinion scores for valence, arousal, dominance and aesthetic quality). The authors also propose EmoProbe, which pairs emotion QA with response-aligned, dimension-specific regression heads. They report that it beats baselines on emotion QA and correlates better with human ratings. The work shows how reliably MLLMs interpret affect in generated images.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Paper presenting the first on-chip, TDC-sensor-based power side-channel attack on a co-tenant FPGA to reconstruct full-scale color ImageNet inputs (224x224) from a victim's first CNN convolution layer, using ML-based upscaling and recoloring. It separates genuinely leaked information from ML-generated post-processing. It also evaluates a pixel-substitution defense based on Hamming Weight, which is largely defeated by an attacker retrained on the transformed data. The authors conclude that robust defenses need randomized or input-dependent transformations.

arxiv.orgpaperAdded 16 Oct 2026, published 12 Oct 2026🛡️ Security

Paper on the forensic analysis of a stand-alone HackRF One with a PortaPack running Mayhem firmware v2.0.1. It establishes at source-code level which data sit on the microSD card, their formats, when they are written and how to interpret them. The method is applied to a four-month SD image and covers RTC-battery-dependent timestamp rules, traces of on-device trimming and text editing, cluster-level recovery of deleted IQ captures, and decoding of recorded signals cross-checked with an independent decoder. In the example, the tail of a deleted IQ capture and a trimmed-away segment are recovered, and six captures are attributed to one rolling-code OOK transmitter with a shared 32-bit ID. It provides an artifact catalogue, interpretation rules and acquisition recommendations.

arxiv.orgpaperAdded 17 Oct 2026, published 11 Oct 2026🛡️ Security

Introduces Confidential Error-correcting Codes (CECs), a primitive that decodes reliably from perturbed codewords while keeping the message confidential, defined by confidentiality, robustness and soundness. The Randomness-Security Separation framework encodes a nonce and masks the payload with a PRF; the Conc-CEC instantiation uses concatenated Reed-Muller and Reed-Solomon codes and claims stronger concrete security than PRCs (CRYPTO'24). The application, CECMark, watermarks AI-generated images and reportedly beats Gaussian Shading, PRCMark and T2SMark across 56 perturbations with a false positive rate below 1e-5 and no side effects.

eprint.iacr.orgpaperAdded 14 Oct 2026, published 10 Oct 2026🧠 AI

Introduces Confidential Error-correcting Codes (CECs), a cryptographic primitive for decoding perturbed codewords while keeping the message confidential, defined by confidentiality, robustness and soundness. The Randomness-Security Separation framework encodes a nonce and masks the payload with a PRF. The Conc-CEC instantiation uses concatenated Reed-Muller and Reed-Solomon codes, and the authors claim stronger concrete security than PRCs (CRYPTO'24). The CECMark application watermarks AI-generated images, and the authors report it beats Gaussian Shading, PRCMark and T2SMark across 56 perturbations with a false positive rate below 10^-5.

eprint.iacr.orgpaperAdded 14 Oct 2026, published 10 Oct 2026🛡️ Security

The paper studies joint attention in multimodal diffusion transformers (MMDiTs), where the evolving image updates text representations. It finds that this image-to-text (I2T) feedback tends to preserve objects and attributes already in the image, errors included, and that suppressing it improves generation. It proposes Semantic Feedback Guidance (SFG), which guides away from an auxiliary branch that strengthens I2T feedback and weakens text-to-image conditioning, and iSFG, an in-place variant that needs no extra model evaluations. SFG runs for the first 20% of sampling steps (about 1.2x base NFE without CFG). Across three backbones, adding SFG to CFG on FLUX raises GenEval from 0.6661 to 0.7479, and iSFG on SD3.5M adds +1.33 HPSv2.1 over CFG at zero extra NFE. The control also improves VBench scores for text-to-video and image-to-video.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

Hide and Seek (HnS) is a concept-level attribution method for black-box vision models. It needs no model weights and no concept-labeled datasets. Given a concept and an image set, it uses a vision-language model to retrieve concept-aligned images and localize the concept, masks the localized region, and takes attribution from the change in the black-box model's output. It agreed better with ground-truth attributions than existing methods on CelebA and COCO-Stuff, and it works on tasks existing concept attribution methods cannot handle, such as image captioning and satellite image segmentation.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

Cryptanalysis of the Alternating Sponge (ASP) and Duet, a permutation-based hash candidate in China's ICCS Next Generation Commercial Cryptographic Algorithm program (NGCC). The authors give an O(2^r)-query distinguisher against ASP that contradicts its claimed indifferentiability bound in some parameter regimes. They also give a distinguisher and state-recovery attack on Duet's multi-block output, using O(2^(r+c-h)) permutation queries. Collision and preimage resistance are unaffected, but Duet used as a XOF or KDF is affected, and state recovery enables length extension.

eprint.iacr.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🛡️ Security

The paper presents RM-KLPT, an algorithm that solves the algebraic pathfinding problem in isogeny graphs of superspecial abelian surfaces with maximal real multiplication. It does this by solving a norm equation in ideals of a quaternion algebra over a real quadratic field. The authors implement it and revisit the RM-CGL hash function of Charles, Goren and Lauter. They show that if the initial surface's algebraic description is known, RM-KLPT gives a second-preimage attack on the hash. Paths are much shorter (~p^3) than with KLPT^2, because a higher-dimensional Deuring correspondence exists in the RM setting. The paper also discusses cryptographic constructions that could be built on RM-KLPT.

eprint.iacr.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🛡️ Security

DreamTrue is a multi-view, cross-embodiment robot world model for action-faithful, physically plausible video prediction. It renders action trajectories into image-space conditions aligned to target videos by offline geometric calibration. Counterfactual post-training modifies recorded trajectories to cover unsuccessful interactions and contact configurations. A human-annotated defect dataset trains an embodied video reward model whose scores drive RL post-training. On AgiBot it reaches state-of-the-art action following and cuts the human-assessed interaction defect rate from 48.12% to 6.25%. It ranked first in the world model track of the AgiBot World Challenge 2026.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

RoboJEPA is an 8B-parameter latent world model built on the Joint Embedding Predictive Architecture (JEPA), trained on real-robot data spanning 12 robotic embodiments. The paper establishes the first scaling laws for multi-embodiment robotic world models, showing that imagination error (latent rollout error) follows a second-order power law in compute and predicts downstream planning performance, and demonstrates zero-shot deployment of the model as a goal-image-conditioned planner on real hardware for long-horizon tasks.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This paper introduces a detector for adversarial images targeting frozen vision-language models (VLMs) by building a 'response profile' from how an image interacts with a bank of general semantic prompts — using category-level statistics, inter-prompt relationships, deviation from clean reference distributions, and stability under weak image transforms, then classifying that compact profile with a lightweight model. Evaluated across multiple public datasets, several CLIP-style visual backbones, and gradient-based, optimization-based, automated, and spatial attacks, the method outperforms embedding-geometry baselines, generalizes partially to unseen attacks, and is tested against detector-aware adaptive attacks.

arxiv.orgpaperAdded 9 Oct 2026, published 7 Oct 2026🛡️ Security

This paper introduces Self-Correction Optimization (SCO), a training-free method that improves consistency in interleaved image-text generation from multimodal large language models. SCO treats the classifier-free guidance update as a reference signal and applies minimal corrections under two constraints—a new-event constraint for temporal consistency and a state-preserving constraint for visual-subject coherence—avoiding the computational cost of additional training with augmented data. The authors show improvements on interleaved multimodal generation benchmarks and extend the method to video generation tasks like robot manipulation and long-horizon handcrafting.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

DictXAI is a new explainable AI method that defines interpretable concepts directly in the input domain using a dictionary of predefined elements, rather than relying on inherently-interpretable input features or architecture-specific internal abstractions as prior XAI methods do. It computes a sparse code of the model input and attributes predictions to dictionary elements, letting it work across learned image bases, analytically-defined ECG waveforms, and experimentally acquired dictionaries, and the authors show it can trace AI malfunctions like Clever Hans effects to specific data artifacts while aiding human-AI alignment on biomedical signals.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

World Models' Last Exam in Physics introduces a measurement-based benchmark of 40 controlled tasks across mechanics, optics, fluids, thermal/phase-change phenomena, electromagnetism, and surface tension, designed to test whether video world models generate physically consistent outputs. Each task pairs an initial image and prompt with predefined physical criteria, and a custom evaluator combines observability screening with task-specific quantitative measurements rather than relying on reference videos or model-based judgments. Testing eight video generation models across 1,280 generated videos found persistent physical inconsistencies, with the best model scoring only 57.76/100, and the evaluator showed higher agreement with human judgments than a vision-language model baseline.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This paper introduces Quantizer-Aligned Recalibration (QuAR), a single-pass, backprop-free test-time adaptation method for quantized vision transformers that recalibrates activations at frozen quantizers to counter distribution shift without updating model parameters. On ImageNet-C with ViT-B across 3-, 4-, 6- and 8-bit weight/activation precision, QuAR beats the strongest backprop-free baselines by up to 4.00 points while cutting latency 46% and adding only 0.17 MB memory overhead, with gains holding across continual streams, label shift, seven OOD suites, and three other backbones.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

IDRF is a reward fine-tuning framework for few-step masked discrete diffusion generators that replaces the intractable sequence-level KL penalty in reverse-KL-regularized objectives with an inverse-distillation regularization term, proven to upper-bound the true KL divergence to the reference distribution. It frames few-step diffusion sampling as a finite-horizon Markov decision process and optimizes reward via a clipped policy-gradient objective over the student's own trajectories, without needing reference-model rollouts. Across DNA, image, and text generation tasks, IDRF reaches high reward with up to 32x fewer denoising steps than the reference model while reducing reward hacking and preserving sample quality.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

DEPICT is a training-free metric for scoring text-to-image alignment that replaces the fixed-YES assumption of decomposed VQA-based evaluators with an expected-agreement score between image-based and caption-only answers, weighted by how decisively the caption determines each question, then merges this with a holistic score. Evaluated across five benchmarks and eleven vision-language model backbones from three model families, it raises negation accuracy from 19% to 88% and outperforms other training-free metrics while beating fine-tuned evaluators on two of three human-correlation benchmarks.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

VisionMX is a post-training quantization method for vision models using Microscaling (MX) formats, which pair low-precision numeric elements with shared block scales for efficient inference. The paper identifies three error sources in direct MX conversion of vision models—block-scale representation, poor alignment of small convolutional weights with nonuniform element grids, and underuse of signed codes by nonnegative activations—and addresses them via bounded weight rounding and a foldable affine activation correction. It is evaluated across image classification, object detection, semantic segmentation, and low-light image enhancement, showing improvements over direct conversion and existing post-training quantization baselines, especially for architectures most sensitive to MX conversion.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

MosaiChunk is a spatio-temporal memory mechanism for autoregressive video generation that composes a "mosaic" of selected historical key-value (KV) cache entries across space and time, letting a frozen video generator recover fine-grained visual details of objects/scenes that fell out of the finite context window. Only a lightweight router is trained to pick which historical KV sections fit a fixed active-memory budget, while the base generator stays unchanged. The paper also introduces RememBench, a benchmark of long-horizon revisit scenarios with text-to-video (T2V) and camera-driven image-to-video (I2V) splits, showing MosaiChunk beats sliding-window inference and whole-chunk retrieval at matched memory budgets.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🧠 AI

Where-OPD introduces an on-policy self-distillation method for multimodal LLMs where a teacher model receives textual spatial guidance (object identities and coordinates from procedurally generated synthetic scenes) to locate relevant image regions, while a student learns to reproduce this behavior from just the image and question. The approach is annotation-free and scalable, improving performance on counting, document and chart understanding, and transferring from synthetic training to real-world perception benchmarks (CVBench, V*, ZoomBench, BLINK, HR-Bench, MME-RealWorld) with a 3.23-point average gain.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🧠 AI

Researchers introduce ImmRAG, a black-box data extraction attack against image-returning Multimodal Retrieval-Augmented Generation (MRAG) systems, where the retrieved image itself is the model's response. Instead of malicious text prompts, ImmRAG embeds attack instructions inside an attacker-supplied image and uses relevance-weighted resampling to steer queries toward unexplored regions of the embedding space, progressively reconstructing more of the private datastore. Tested against CLIP-family retrievers across medical, document, and general-purpose scenarios, a single 2500-query run reconstructed up to 611 radiology images, 566 document scans, and 416 general images — up to 5.6x more distinct items than a non-adaptive baseline — highlighting the lack of safeguards against multimodal data leakage.

arxiv.orgpaperAdded 2 Oct 2026, published 1 Oct 2026🛡️ Security

Multimodal Flow (MF-1) is a fully continuous generative model that unifies language and vision modeling by representing both text and images as ordered continuous "hyperchunks" processed through a single shared chunk-causal Flow Matching backbone, avoiding the visual quantization bottleneck of discrete-token unified models and the mismatched objectives of hybrid discrete/continuous approaches. Joint attention handles cross-modal interaction while modality-specific feed-forward networks process each modality, and the model predicts multiple chunks in parallel during training but generates sequentially at inference. Pretrained at 0.6B/1.2B/1.6B scales on 150B tokens, MF-1 scores 82.8 average on GenEval/DPG-Bench and 75.3 on VQAv2/MMBench/POPE, remaining competitive with unified models trained on far more data and outperforming matched hybrid/discrete baselines; code and model are publicly released.

arxiv.orgpaperAdded 1 Oct 2026, published 30 Sep 2026🧠 AI

AssemblyWorld is an interactive 3D environment where general-purpose agents assemble objects (furniture, industrial parts, fractured pieces) by inspecting rendered 2D views and manipulating supplied rigid parts, optionally guided by images or assembly manuals, without any assembly-specific fine-tuning. The accompanying AssemblyWorldBench benchmark covers 100 tasks across 80 objects and evaluates eight agent systems geometrically; the best system reaches 80.9% part accuracy but only 59.4% complete-assembly success, and open-source systems lag closed-source ones in both reliability and accuracy. The paper analyzes interaction trajectories and failure modes to characterize the gap between approximate structure recovery and precise 3D reconstruction.

arxiv.orgpaperAdded 1 Oct 2026, published 30 Sep 2026🧠 AI

This paper introduces VideoMSN, a Masked Siamese Network that repurposes standard image Vision Transformers for self-supervised spatio-temporal video representation learning by representing videos as 'super images' (grids of sampled frames) and aligning spatial-masked and temporal-masked views with a shared ViT encoder. Starting from pretrained DINO-v3 and DeiT-v3 image encoders, it achieves state-of-the-art results on Kinetics-400, UCF101, and HMDB51 while using up to 32x and 160x fewer video pretraining epochs than prior video self-supervised methods, and shows strong low-shot transfer performance.

arxiv.orgpaperAdded 1 Oct 2026, published 30 Sep 2026🧠 AI

Point2Part is a promptable 3D part-decomposition model that, given 3D point prompts (one per desired part), jointly partitions an image or mesh into non-overlapping parts that fully cover the shape, unlike prior methods that allow overlaps or gaps. It builds on a pretrained 3D generation model's shape latent space, adding a prompt encoder mapping point prompts to part tokens and a coarse-to-fine part decoder that assigns every shape position to exactly one part, unifying image-to-part generation, mesh-to-part generation, and part segmentation in one model. The authors report outperforming prior SOTA on part-quality metrics and an order-of-magnitude improvement in part compatibility across all three tasks.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

Imagine3D-LLM appends learnable summary tokens after image tokens in an MLLM, decoding them into a compact 3D Gaussian Splatting representation supervised by a photometric reconstruction loss, trained jointly with next-token prediction. This mimics human coarse spatial reasoning (identifying common objects across views, inferring relative geometry) rather than relying on fine-grained pixel correspondence or fused 3D geometry foundation model features, and the paper shows this reconstruction objective also improves cross-frame correspondence in the LLM's image features. The approach outperforms prior methods across multiple spatial reasoning and 3D understanding benchmarks.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

This paper shows that adding an adversarial loss as a post-training correction to pretrained pixel diffusion/flow-matching models (applied at non-high-noise timesteps, architecture and sampler unchanged) restores missing high-frequency, fine-scale natural-image statistics that these models systematically underproduce. Through frequency-band/power-law analysis and nearest-neighbor, recall, and no-GAN matched-SFT controls, the authors rule out memorization or mode-dropping as explanations, and show the effect jointly improves distribution fidelity, coverage, prompt alignment and perceptual quality — but does not transfer comparably to latent diffusion configurations, pointing to direct pixel-output access as the key enabling factor.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

DMA^2 is a distribution matching distillation (DMD) method tailored to pixel-space (rather than latent) diffusion teachers for text-to-image generation. It introduces DINO-Adv, which removes the heavy fake-score critic from the adversarial gradient path and instead supplies local patch guidance via an external DINOv2 representation, plus AF-Loss, a parameter-free auxiliary semantic distribution-field objective operating in DINOv2 feature space on rolling real/generated supports. The resulting four-step student outperforms its 25-step teacher and other few-step distillers on DPG-Bench, GenEval, VQAScore, and COCO30K.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

FracGen is a fracture-aware video generation model that, given a single image of an intact object, generates plausible and controllable fracture dynamics (where an object tears, crack propagation speed, pre-failure deformation) conditioned on physics signals. It is trained using FracSim, a material point method (MPM) simulation framework augmented with a continuum damage model that produces paired fracture videos and dense pixel-aligned physical fields at no extra rendering cost; FracGen jointly predicts these fields alongside RGB video and is supervised with physics-informed consistency losses. The authors also introduce a benchmark for evaluating physical plausibility of generated fracture video, showing FracGen outperforms existing video generation baselines in both physical and visual fidelity.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

This paper shows that internal expert-routing probabilities in Mixture-of-Experts vision-language models (Qwen3-VL-30B-A3B-Instruct and Gemma-4-26B-A4B-it) carry a strong, low-cost signal for detecting when a model is about to answer a question about an object that isn't actually in the image (a 'target-absence grounding failure'). Linear detectors trained on target-token routing probabilities achieve ROC-AUCs of 0.9988/0.9956 on GQA-Inpaint (retaining 0.78-0.81 on the external OBER dataset), and gating a target-aware review prompt on these detections improves end-to-end accuracy by up to +22% without modifying model weights. The work matters for practitioners building on MoE VLMs because it demonstrates a weight-free, pre-generation hallucination detector that reuses computation the model already produces.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

FurE is a strand-based method for reconstructing editable 3D animal fur from multi-view images without requiring animal-fur training data. It optimizes a root-conditioned latent field decoded into strand geometry via a PCA-based decoder trained on human-hair strand data, combined with a defurred animal body recovered using a surface-constrained Gaussian Frosting representation and part-based priors, achieving a 10x speedup over dense per-strand optimization baselines while retaining strand fidelity on synthetic and real-world sequences.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

UMM-Reflection is a reinforcement learning method that trains a unified multimodal model to jointly learn reflection text and image revision within a single closed loop: diagnose an image's flaws, revise it, observe the result, and repeat. Sibling trajectories sharing one initial image enable a group-relative advantage that compares whole reflection strategies rather than assigning credit per round, and one trajectory-level advantage updates both the reflection tokens and the flow-based revision head jointly, unlike SFT cold-starts or RL that optimizes only the renderer. On BAGEL, it improves GenEval by 12.05 points over SFT and transfers to unseen benchmarks WISE, OneIG-Bench and T2I-CompBench++, with no external verifier needed at inference.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This arXiv paper introduces a unified theoretical framework for "distributional training" in one-step visual generation, showing that existing methods like FD-Loss and Gaussian-kernel Drifting are special cases of Gaussian optimal transport and kernel-density KL matching under Wasserstein gradient flow. Building on this, the authors propose MGFlow, which models feature distributions with Gaussian mixtures at adjustable granularity and combines mass-constrained sample assignment with paired component updates to fix mode collapse. MGFlow sets new state-of-the-art FDr scores on ImageNet 256×256 (1.45 on pMF-H, 1.64 on JiT-H) and post-trains FLUX.2 into a one-step generator that beats the original four-step model on GenEval and PickScore.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

FlowAct-R2 is a framework for interactive humanoid video generation that pairs a Streaming Multimodal Reference Diffusion Transformer (built on the Seedance 2.0 Mini reference-to-video backbone) with a Proactive Interaction Agent. The diffusion component accepts rolling action prompts plus continuously updated image, audio and video references, using video-driven rotary positional embeddings and partially noised historical motion frames to keep appearance consistent and avoid drift over long generation runs. The agent component separates offline persona/agenda/skill preparation from online scheduling, audience response and interruption handling, enabling real-time 720p, hour-scale streaming avatars for use cases like live shopping, video chatting and live vlogging.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

MEOW is a feed-forward 3D reconstruction model that jointly predicts metric pointmaps and camera poses from N-view image tuples mixing perspective, fisheye, and full 360° panoramic images in a single forward pass, without any camera calibration, distortion parameters, or camera-type labels supplied. It keeps a perspective-pretrained backbone and adapts it to heterogeneous cameras purely through a procedural synthetic data engine that renders scenes across a continuous manifold of camera models with exact rays, depth, and verified covisibility, then transfers zero-shot to real captures, beating the Wid3R baseline (79.9 vs 53.8 mAA@30 on heterogeneous 2D3DS tuples even when Wid3R is given ground-truth camera types) and scoring 79.4 AUC@30 on a new laser-scanned mixed-camera benchmark.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

DeepSeek Elastic Compute (DSec) is a production sandbox platform that unifies FnCall, container, microVM, and full-VM execution backends behind a single SDK to support large-scale agentic training and evaluation of LLMs. It co-designs sandbox lifecycle management with the RL training framework — decoupling stateful rollout execution from preemptible GPU training, sharing and reclaiming memory, and streaming images on demand from DeepSeek's Fire-Flyer File System (3FS) — to run around 160 nodes serving ~3 million sandboxes/day, over 380,000 concurrent sandboxes, and 5,000+ sandbox creations per second in production.

arxiv.orgpaperAdded 2 Oct 2026, published 26 Sep 2026🧠 AI

LOGOS is a proposed pluggable governance and self-evolution layer that sits atop existing multiagent frameworks, compiling documents, images, audio, tables, databases, APIs and human instructions into versioned "agent packs" containing agents, tools, knowledge, tests, permissions and policies. It converts agent activity into portable, auditable event traces and enforces fail-closed verification, requiring held-out execution evidence and human authorization before any learned prompt, memory, skill, tool, role or workflow can be promoted from candidate to production. The core contribution is architectural: a framework for "verifiable human-agent loop engineering" that lets agents act and propose improvements at speed while humans retain control over irreversible changes.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

Researchers report that attaching an unrelated "decoy" image to an encoded jailbreak prompt can sharply cut attack success rate against caption-mediated black-box VLM defenses like ECSO — up to 73 percentage points — even though the image itself is content-free, across five frontier VLMs, two encoded-attack families, and three defenses. Controls show the effect stems from image presence (not content or vendor filtering) and holds even under adaptive attacks targeting the caption re-check, but unconditional decoy attachment inflates benign refusal rates by 10–67pp, so the authors propose gating it on a lightweight encoded-input detector. The paper was withdrawn by the author after v2, so treat conclusions as unverified/retracted.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This paper introduces QVec, a defense against Quantization-Conditioned Backdoors (QCBs) — a threat where a neural network behaves normally in full precision but activates hidden malicious behavior only after quantization. QVec treats the weight difference between the full-precision and quantized model as a "malicious task vector" (via task arithmetic) and applies a lightweight parameter correction before deployment to cancel it out, requiring no retraining, no trigger samples, and only one quantization pass plus a small hyperparameter search. It's evaluated across image classification benchmarks and multiple LLM backdoor attack scenarios, suppressing backdoor activation while preserving clean-model performance.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

FuseReg is a training method for representation autoencoders (RAEs) that replaces fixed heuristic selection of pretrained-encoder layers with training over random subsets of encoder layers, explicitly regularizing sensitivity to cross-layer disagreement. On ImageNet-256 with DINOv3-L, a single FuseReg decoder matches or beats decoders specialized to fixed layer fusions across full, sparse, and single-layer configurations, and swapping it into an unchanged RAEv2 DiT-XL generator cuts unguided gFID by 27%; joint regularization during diffusion training further reduces unguided gFID by 29% on DiT-Base. The work targets the trade-off where shallow encoder layers preserve pixel detail but deep layers yield better generation metrics, narrowing the reconstruction-generation gap without modifying the pretrained encoder.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

This arXiv paper addresses aligning the distribution of an attribute in a black-box generative AI model's outputs, such as fairness attributes or synthetic data representativeness, with a user-specified target distribution, using only repeated query access without model internals. The authors develop query-minimizing post-processing algorithms for exact and approximate alignment across multiple generated outputs, proving asymptotic optimality, and validate the approach on text-to-image generation and geocoded persona generation tasks.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

This paper presents a two-layer humanoid locomotion architecture: a perceptive flow-matching motion generator that plans whole-body trajectories from raw depth images, paired with a control-guided RL tracking policy that follows those motions. The core contribution is a sample-efficient off-policy RL fine-tuning loop — using structured search with the generator plus advantage weighted regression — that improves terrain consistency on unseen geometries, boosting successful terrain traversals by up to 25 percentage points and skill selection by up to 80 points. The system is deployed on a Unitree G1 humanoid, enabling walking, running, jumping on/off boxes, and stair traversal outdoors using only raw depth images (no odometry or height maps).

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

CLIPGuard is a fully black-box defense against embedding-space backdoor attacks in CLIP, which poison a tiny fraction of image-text pairs to implant stealthy triggers that shift the joint embedding space without needing model parameters, gradients, or clean validation data. It detects malicious image regions by measuring segment-wise embedding perturbations and purifies only suspicious segments via semantic inpainting, reducing attack success rate to 1.05% while keeping clean accuracy at 86.34% on STL-10 and ImageNet against BadCLIP, BadNets, blended, patch-based, and typographic triggers, outperforming CleanCLIP and CleanerCLIP.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🛡️ Security

PoEM predicts the outcome of RL post-training on a new reward function using models already trained on other rewards, without running additional RL. It shows that when a new reward is a linear combination of existing ones, the resulting log-policy is a corresponding linear combination of existing log-policies, and even for non-linear reward combinations, log-policies often span a low-rank subspace whose combination weights can be estimated from sample outputs alone. Validated on synthetic and real rewards across text and image modalities.

arxiv.orgpaperAdded 26 Sep 2026, published 24 Sep 2026🧠 AI

This paper studies Representation Autoencoders (RAEs), which let diffusion models generate images in the feature space of pretrained visual encoders. It shows that finetuning encoders for better reconstruction reduces the effective dimensionality of their representations, which makes standard velocity-prediction flow matching inefficient because the model must fit noise directions orthogonal to the low-dimensional signal manifold; switching to x0-prediction (clean data parameterization) instead consistently improves text-to-image generation across multiple strong-reconstruction encoders.

arxiv.orgpaperAdded 24 Sep 2026, published 23 Sep 2026🧠 AI

This paper tightens the soundness bound of the compressed permutation oracle technique (a proof tool for analyzing quantum adversaries against random-permutation/ideal-cipher-based constructions) from O(N^1/12) queries to a tight Ω(N^1/2), using a POVM-based construction of the compression isometry. The improved bound yields concrete quantum collision and preimage lower bounds for the sponge construction underlying SHA3 and the Davies-Meyer compression function used in SHA1 and SHA2, and is offered as a general tool for quantum security proofs involving random permutations or ideal ciphers.

arxiv.orgpaperAdded 24 Sep 2026, published 23 Sep 2026🛡️ Security

HARMONY is a hierarchical chain-of-thought framework for reconstructing full 3D indoor scenes from a single monocular image, combining agentic VLM reasoning for spatial/semantic layout with visual geometry foundation models for point-cloud-based refinement. It calibrates camera pose, infers room layout and object placement order via VLM reasoning, then places objects hierarchically (wall-mounted, free-standing furniture, then decorations) using depth-first traversal and a reflective feedback loop to limit error accumulation. Experiments on synthetic and real images show it outperforms reconstruction baselines and produces more faithful, image-aligned object arrangements than comparisons with GPT-6 Astra.

arxiv.orgpaperAdded 23 Sep 2026, published 22 Sep 2026🧠 AI

WorldCrafter is a video world model that learns a camera-queryable implicit 3D-aware memory to preserve consistency across long horizons and viewpoints during interactive scene exploration. A memory encoder and pose-conditioned readout module, trained jointly with the video generator, compress multi-view historical observations into fixed target-view-specific tokens without explicit depth-based correspondences, and combined with few-step distillation enable streaming exploration from a single image or text prompt with improved long-horizon consistency and camera-control accuracy.

arxiv.orgpaperAdded 22 Sep 2026, published 21 Sep 2026🧠 AI

Paint-Anything introduces a shared hex-color prompt interface for both image generation and editing, letting users specify an object's exact 24-bit hex color. The authors build the Paint-500K dataset via object grounding, perceptual color labeling, and editing-pair synthesis, and supplement real-image supervision with pure-color anchors used only at high-noise diffusion timesteps to counter shadow-induced label imprecision. Evaluated on FLUX.2-4B with the new Any Color Benchmark (ACBench-T2I/ACBench-Edit), it improves color-fidelity scores by 85.3% and 28.3% respectively over the base model.

arxiv.orgpaperAdded 22 Sep 2026, published 17 Sep 2026🧠 AI

This paper proposes a hybrid generative model that combines denoising diffusion with an adversarial loss (replacing the standard denoising loss, since no 3D ground truth exists) to synthesize 3D geological/material microstructures from 2D image slices. It targets heterogeneous clay and cementitious materials, where the prior GAN-based approach SliceGAN struggles to capture fine detail, and shows the new model produces volumes with fewer slice artefacts and closer agreement to ground-truth phase fractions and structural descriptors.

arxiv.orgpaperAdded 22 Sep 2026, published 17 Sep 2026🧠 AI

PANORAMA is a vision-language model for panoptic grounded captioning: it generates dense image captions while grounding each referring phrase to pixel-level masks covering both foreground objects and background regions. The authors formulate phrase grounding as selection from a phrase-conditioned pool of mask proposals produced by conditioning a pretrained segmenter on contextualized phrase representations, trained jointly with caption generation. They also introduce PanoCaps, a human-annotated benchmark with near-complete pixel coverage and entity-level image-text alignment, plus a generalized Panoptic Quality (gPQ) metric jointly evaluating text and mask agreement; PANORAMA achieves state-of-the-art grounding on PanoCaps and matches specialized models on several pixel-level grounding tasks.

arxiv.orgpaperAdded 22 Sep 2026, published 16 Sep 2026🧠 AI

PhysStream is an autoregressive image-to-video generation model that maintains structured scene memory (positional maps and object tracking maps derived online from previously generated frames) and supports fine-grained, mid-generation motion control via sparse velocity-increment signals encoding physical quantities. Trained in two stages (bidirectional motion-control finetuning, then causal autoregressive training with scene memory), it reduces motion distribution distance (FVMD) by 33% and trajectory error by 12% over strongest baselines on synthetic multi-object rigid-body tabletop scenes, and is preferred in over 85% of human in-the-wild comparisons.

arxiv.orgpaperAdded 22 Sep 2026, published 15 Sep 2026🧠 AI

SenseNova-U1.5 is an 8B-parameter Mixture-of-Transformers native unified multimodal model that understands, reasons about, and generates visual content within an encoder-free and VAE-free architecture. It uses spatially coherent patch reconstruction, native 4K resolution training, and post-training with specialized experts (aesthetics, bilingual text rendering, infographics, image editing) consolidated via multi-expert on-policy distillation. The paper reports advances in image fidelity, text rendering, complex composition, and multi-reference editing, and plans to open-source training code including SFT, RL, and on-policy distillation.

arxiv.orgpaperAdded 22 Sep 2026, published 10 Sep 2026🧠 AI

This paper evaluates whether small, edge-deployable vision-language models (Qwen3-VL 2B/4B/8B, Gemma3 4B) in the 2-8B parameter range possess genuine taxonomic knowledge for species identification, comparing them against the 300M-parameter domain-specialist BioCLIP on a 96-species task. Testing clean iNaturalist photos against real camera-trap imagery from 6 field collections, they find all models degrade sharply (9.6-26.6 point domain gaps) on field imagery, that BioCLIP outperforms every VLM by 33-59 points despite its smaller size, and that under open-set prompting 5.9-9.6% of model responses are fabricated, taxonomically nonexistent species names with consistent per-model fabrication rates across evaluation sets.

arxiv.orgpaperAdded 22 Sep 2026, published 10 Sep 2026🧠 AI

This paper introduces a training-free framework for incorporating partial geometric observations (e.g. depth or point measurements) at test time into pretrained image-to-3D generative models, without retraining or finetuning. It guides generation via a ray-consistent observation likelihood over the model's occupancy representation, combining surface occupancy and free-space evidence, applied to SAM 3D and its multi-view extension to improve geometric fidelity and visual quality.

arxiv.orgpaperAdded 22 Sep 2026, published 9 Sep 2026🧠 AI

Point4D is a feed-forward model for 4D reconstruction of long-range video sequences, capable of reliably tracking dense per-point 3D trajectories across multi-hundred-frame videos where prior methods were limited to a few dozen frames. Its key innovation is a flexible 3D query-based motion decoder that decouples trajectory prediction from image-plane visibility, re-querying predicted 3D endpoints directly in the next chunk without re-projection or matching, plus reuse of visual descriptors from arbitrary visible frames rather than only the source patch. It achieves state-of-the-art results on long-video tracking benchmarks spanning 200+ frames, substantially outperforming prior feed-forward 4D methods.

arxiv.orgpaperAdded 22 Sep 2026, published 8 Sep 2026🧠 AI

Diffusion TV is an interactive AI art installation built from a modified CRT TV that lets audiences control the clarity of AI-generated images and sounds by physically manipulating the antenna, metaphorically enacting the denoising process of diffusion models. A tuning knob switches between three channels of AI-generated animals (extinct, endangered, speculative), embedding the interaction in a temporal and ecological narrative. The work argues for embodied, tangible interaction as an alternative mode of explainable AI that foregrounds the generative process itself rather than final outputs.

arxiv.orgpaperAdded 22 Sep 2026, published 4 Sep 2026🧠 AI

Puffin-World is a unified multimodal architecture that jointly models physics (gravity/latitude), geometry (depth), and appearance (image) as native 3D world states, using an Omni-Camera representation to support 3D world generation, reconstruction, and physically consistent simulation without external offline modules. The authors also release Puffin-16M, a dataset of 15M vision-language-camera triplets and 1M motion trajectories, along with code and models to support closed-loop world exploration tasks.

arxiv.orgpaperAdded 22 Sep 2026, published 3 Sep 2026🧠 AI