54 results for “Alignment”

GPT-Rosalind-6 System Card Addendum describes OpenAI's life-sciences reasoning model, which uses the same weights as GPT-6 Astra and is offered only through trusted access. It reports life-science benchmark gains over GPT-Rosalind-5.5 (MedChemBench v7 49.6% vs 28.9%), High-capability Biological and Chemical Preparedness results, and below-threshold Critical bio design evaluations. It also covers access controls, refusal evaluations, cyber jailbreak regression and alignment tables.

deploymentsafety.openai.compaperPublished and added 30 Oct 2026🧠 AI

MedCBR is a two-stage alignment framework for medical vision language models. Difficulty Weighted Boundary Preference Optimization (DBPO) trains on counterfactual preference pairs at the model's capability frontier. Entropy Gated Grounded Policy Optimization (EGPO) then applies distilled clinical rubrics according to prediction entropy. It reaches 61.2% average accuracy on six medical benchmarks, with the largest gains on differential diagnosis, and the two stages combine super-additively.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

Addendum to the GPT-6 Astra system card for OpenAI's GPT-6.1 Terra, rated Critical in cybersecurity and High in biological and chemical capability under the Preparedness Framework. It reports multiturn jailbreak and prompt-injection results, alignment tests (auto-review circumvention, ExploitGym honeypot, monitor evasion), and cyber scores: 97.10% ExploitBench, 24.5% on the internal recent-vulnerability port, 5 of 22 Sandbox Bench targets.

deploymentsafety.openai.compaperPublished and added 26 Oct 2026🛡️ Security

This paper studies demographic bias in LLM-emulated electorates, using three U.S. presidential elections and comparing one closed-source and one open-source LLM. Removing gender from the voter profile cuts the largest group error by 22-42%, and removing education cuts it by 64%. Removing self-reported race raises mean group error by over 40%. Repeated queries and fairness instructions do not reduce errors and can double them.

arxiv.orgpaperAdded 27 Oct 2026, published 23 Oct 2026🧠 AI

Sentiment-Injected Sentence Embeddings (SISE) addresses a finding across eleven frozen text encoders (2021 to 2026): linear probes recover sentence polarity at up to 91.3% accuracy, but cosine similarity separates polarity by only 0.003-0.064. A small supervised-contrastive head (0.13M-0.69M parameters) trained on CPU raises retrieval MAP by up to 26 points, but the 128-d head loses 38-41 STS-B Spearman points. The paper also releases the Attribute Alignment Gap diagnostic.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

Sensorimotor Alignment (S^2) is a training-free offline metric that measures how well an embodied model's sensory representation aligns with the geometry of physical motion. It extends the Platonic Representation Hypothesis to motion trajectories. S^2 correlates positively with downstream robot task success across embodied model families, helps pick promising VLM backbones before fine-tuning into VLAs, and explains behaviours seen in VLAs and world action models.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

This paper studies evidence-grounded health summarization for menstrual health monitoring. It combines structured evidence conditioning with RAG in a prompt-based constrained generation framework, and scores 4,500 summaries from three LLMs with NLI, atomic factuality metrics and readability measures. Grounding constraints improved uncertainty preservation and generally kept or improved faithfulness against retrieved evidence, while omission effects varied by model. Clinical alignment came with harder reading and more medical terminology.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

A Spectral Theory of Local-to-Global Leakage is an IACR ePrint paper (2026/2571) by Rei Ueno on additive secret sharing over finite abelian groups. It gives an exact leakage decomposition into local leakage magnitude and spectral alignment across shares. Two channels with identical local mutual information can have exponentially different global leakage. Automorphism-weighted sharing can cancel leakage perfectly with finitely many shares, which ordinary additive sharing cannot do.

eprint.iacr.orgpaperAdded 21 Oct 2026, published 20 Oct 2026🛡️ Security

Beacon is a safety alignment framework for large reasoning models that fixes two GRPO failure modes: advantage collapse, where homogeneous rollout groups give vanishing advantages, and unsafe diversification, where unsafe behaviors persist and spread. It combines Safety Signal Recovery (advantage-guided trajectory filtering plus safe reference supervision) with Safety-Utility Decoupled Alignment (category-aware rewards, benign-only KL anchoring). Tested on four Qwen3 and DeepSeek-R1-Distill variants across eight benchmarks, it cuts average safety error by up to 32.83 percentage points with low over-refusal.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

PixelMoE is a pixel-space diffusion framework that scales capacity sparsely using standard LLM-style Mixture-of-Experts: token-choice routing, fine-grained routed experts, a shared expert and bias-based load balancing, with no diffusion-specific routing or auxiliary routing losses. It adds Pixel-Space Hierarchical Flow Matching, which varies spatial resolution along the denoising trajectory from global structure to fine detail. At matched active parameters it beats dense counterparts, and reaches FID 1.47 at 256x256 and 1.53 at 512x512 on class-conditional ImageNet-1K without representation alignment or perceptual losses. Code and models are promised but not yet released.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

RARA (Retrieval-Anchored Representation Alignment) is a two-stage method for adapting pre-trained robot policies such as Vision-Language-Action models to new tasks from few demonstrations. It first retrieves pretraining representations with similar task intent and motion and aligns the policy's representations of the target demonstrations to them. It then fine-tunes on the target demonstrations while constraining representation change on the retrieved demonstrations, which limits representation drift. Tested with Diffusion Policy and GR00T on 13 tasks in RoboTwin, RoboCasa and on a real robot, it improves average success over plain fine-tuning by 9.5 points in-distribution and 13.6 out-of-distribution, and beats all compared baselines.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

A narrative review in HCI arguing that the single-voice chatbot interface causes three harms: cognitive offloading, ideological narrowing of user beliefs, and epistemic compression of cultural and value diversity. It proposes multi-persona interfaces, which show several personas to the user at once, as a low-cost fix. The design draws on game design and participatory simulation (magic circle, role differentiation, persona embodiment). It offers a taxonomy for designing such interfaces and calls for empirical studies that link technical pluralistic alignment methods to real human-AI interaction.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

OmniCapBench is a benchmark of 786 densely annotated videos that evaluates audio-visual captioning in multimodal LLMs. Instead of scoring free-form captions, it asks models for sets of atomic, verifiable units across three tracks (entity references, visual shots, audio events). These are scored with deterministic constraint checks plus localized LLM-based semantic comparisons, which gives coverage, localization and stable scoring together. Tests on frontier MLLMs show strong local perception but weak long-horizon audio-visual reasoning, especially identity drift and cross-modal misalignment.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

ReSI is a recursive safety improvement framework that automates rounds of safety alignment. Each round applies diverse red-teaming methods to the current target model, develops training recipes to fix the vulnerabilities found, and promotes the update with the largest safety gain among those passing a Pareto gate on capability retention. On four dense and mixture-of-experts models it matches or exceeds frontier models on in-distribution and out-of-distribution safety benchmarks. It cuts mean X-Teaming attack success rate from 86.01% to 31.45%, against 56.69% for the best frontier model (GPT-5.6 Luna), while largely preserving general capabilities.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

ReSI is a recursive safety improvement framework. Each round runs diverse red-teaming methods against the current target model, develops training recipes for the vulnerabilities found, and promotes the update with the largest safety gain among those passing a Pareto gate on capability retention. Across four dense and mixture-of-experts models it cuts mean X-Teaming multi-turn jailbreak attack success from 86.01% to 31.45%, versus 56.69% for the best frontier model evaluated (GPT-5.6-Luna). It also beats alignment baselines on nearly all safety evaluations while largely preserving general capability. It frames the goals as resistance to known threats and resilience to unforeseen ones, following R²AI.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🛡️ Security

TRACE (Train-Rollout Quantization Alignment via Compact GuidancE) is an FP4 quantization framework for reinforcement learning post-training of Mixture-of-Experts language models. It uses rollout-guided quantization-aware training to align training-side FP4 rounding with rollout-side quantization outcomes, reducing the train-rollout discrepancy that limits prior low-precision RL methods, and adds a caching scheme that selectively retains mantissa/scale information from deeper layers to cut communication overhead. Evaluated on four large MoE models across reasoning, coding and long-horizon RL tasks, it matches BF16-rollout RL performance while achieving up to 5.4x rollout speedup with joint FP4 weight/activation and FP4 KV-cache quantization.

arxiv.orgpaperAdded 7 Oct 2026🧠 AI

TaoD2C-Bench is a benchmark for evaluating multimodal large language models (MLLMs) on industrial design-to-code (D2C) tasks — converting UI designs into code that satisfies implementation constraints, not just visual fidelity. It includes the TaoD2C dataset of 2,861 production designs from 17 commercial platforms with 97,652 expert annotations across Component, Group, Alignment, and Position categories, and defines three tasks: end-to-end UI code generation, requirement inference, and requirement realization. Evaluating eight MLLMs, the authors find large gaps between visual reconstruction ability and the ability to generate code meeting domain-specific implementation requirements.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

DictXAI is a new explainable AI method that defines interpretable concepts directly in the input domain using a dictionary of predefined elements, rather than relying on inherently-interpretable input features or architecture-specific internal abstractions as prior XAI methods do. It computes a sparse code of the model input and attributes predictions to dictionary elements, letting it work across learned image bases, analytically-defined ECG waveforms, and experimentally acquired dictionaries, and the authors show it can trace AI malfunctions like Clever Hans effects to specific data artifacts while aiding human-AI alignment on biomedical signals.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv preprint argues that proofs of AI safety or alignment which consider the agent in isolation are incomplete, because the environment's structure matters just as much. The authors construct two theoretical scenarios — one with an infinite state space, one with signal mixture — under partial observability and time-limited action, and prove that even a hypothetically perfect agent cannot guarantee safe behavior in them. The conclusion is that any rigorous safety or alignment proof must jointly specify the agent and the environment's safe-action structure, not the agent alone.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This paper argues that training "model organisms" (models deliberately given alignment-relevant behaviors like backdoors, sycophancy, or spurious correlations) to a single behavior-installation objective produces unrealistic test subjects for interpretability research. The authors propose a three-objective validation framework (behavior installation, general-capability preservation, output naturalness) and show chat quality and chain-of-thought naturalness degrade substantially across common training recipes, with these degradations predicting how well interpretability methods such as logit lens recover the installed behavior. They introduce a model-merging-based multi-objective training approach and apply it to a new model-organism suite for demographic bias in clinical reasoning, finding DPO training preserves the base model better than supervised finetuning.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv paper proposes foundations for proactive LLM agents organized around three joint principles (3T: Task Capability, Temporal Allocation, Trust) and a five-dimensional design space (task scope, anticipation horizon, activation trigger, processing timing, intervention depth). It introduces PROACTIVITY-GYM, a simulation-based testbed with multi-day scenarios, stateful environments, and persona-conditioned simulated users, and reports evaluations across 23 model-harness configurations plus a 30-participant human study showing sharp trust declines after intervention misalignment and a preference for imperfect sleep-time assistance over interruptive help.

arxiv.orgpaperAdded 6 Oct 2026🧠 AI

This paper studies robust bandits (formerly "imprecise bandits," Kosoy 2025; Appel and Kosoy 2025), a framework for unrealizable learning in the bandit setting where the true environment may lie outside the learner's hypothesis class. While prior work showed a Θ(√T) regret learner exists for a broad class without computational guarantees, this paper identifies a special case admitting a polynomial-time learner achieving Õ(√T) regret, and proves several natural generalizations of that case are NP-hard — mapping the boundary of computational tractability. The authors frame this as relevant to AI alignment, following Kosoy's argument that efficient learners for unrealizable problems are a prerequisite for solving alignment.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This paper proposes a formal framework for "alignment scaling laws," modeling the compute/capability burden needed to hold a fixed safety target for each risk category as a power law B_r(N)=a_rN^alpha_r, where the exponent alpha_r determines whether alignment gets easier, keeps pace, or accumulates "alignment debt" as capability N grows. The authors prove theoretical results (the largest exponent among corrected risks sets the long-run regime, super-exponential headroom requirements above alpha=1, small-model fits underestimate large-scale exponents for power-law mixtures) and apply a preregistered protocol twice: a reanalysis of adversarial-training data on Pythia classifiers (attack-success compute exponent 0.60) and a pilot across Qwen2.5 (0.5B-72B) and Qwen3 (0.6B-14B), finding scaling helps for truthfulness and stated dispositions but sycophancy (exponent ~0.83-0.89) and a planted backdoor behave worse or remain undetermined, with the backdoor surviving blind safety training at most model sizes. The work matters as an early attempt to put the "does alignment get harder with scale" debate on measurable, falsifiable footing rather than anecdote.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

LESSER is a drop-in wrapper for gradient-based post-training data selection that approximates full-parameter gradient alignment using only output-layer gradients, which can be computed with a forward pass instead of a full backward pass. It cuts feature-extraction compute by 9.7x on SFT benchmarks and 3.0x on RL benchmarks while matching the downstream performance of full-gradient selection methods, even though individual sample rankings differ, selected batches remain gradient-aligned.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

The World Embedding Benchmark is a new evaluation suite of 8,000 controlled simulation cases across 80 families (fluid mechanics, solid mechanics, dynamics, optics & electromagnetism), each pairing a rendered video with simulation-derived physical annotations. It tests pre-trained omnimodal embedding models on text-video retrieval, physical-property regression, and multiple-choice video-description classification, finding that current embeddings align poorly cross-modally while lightweight probes can still recover physical quantities, and that continual contrastive training improves alignment but hurts regression. The authors also show retrieval-augmented generation using these embeddings with MiniMax-H3 improves the physical fidelity of generated videos.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

DEPICT is a training-free metric for scoring text-to-image alignment that replaces the fixed-YES assumption of decomposed VQA-based evaluators with an expected-agreement score between image-based and caption-only answers, weighted by how decisively the caption determines each question, then merges this with a holistic score. Evaluated across five benchmarks and eleven vision-language model backbones from three model families, it raises negation accuracy from 19% to 88% and outperforms other training-free metrics while beating fine-tuned evaluators on two of three human-correlation benchmarks.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

CLIMB is a training-free, inference-time framework for multimodal retrieval-augmented generation that improves knowledge-intensive visual question answering. It first builds a compact, non-redundant evidence pool using an MMR-style objective balancing query relevance and redundancy, then applies an R/E/C critic (relevance, evidence specificity, cross-modal alignment) alongside a confidence estimator that only accepts an answer update when estimated confidence increases, giving a built-in stopping criterion without modifying the underlying retriever or MLLM. Experiments on Encyclopedic-VQA and InfoSeek show consistent gains over existing multimodal RAG baselines.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

VisionMX is a post-training quantization method for vision models using Microscaling (MX) formats, which pair low-precision numeric elements with shared block scales for efficient inference. The paper identifies three error sources in direct MX conversion of vision models—block-scale representation, poor alignment of small convolutional weights with nonuniform element grids, and underuse of signed codes by nonnegative activations—and addresses them via bounded weight rounding and a foldable affine activation correction. It is evaluated across image classification, object detection, semantic segmentation, and low-light image enhancement, showing improvements over direct conversion and existing post-training quantization baselines, especially for architectures most sensitive to MX conversion.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

This paper studies whether forcing an LLM to reason in a specific language helps or hurts accuracy in retrieval-augmented generation (RAG), extending prior findings from short-prompt reasoning to settings where the model must integrate large retrieved context. The authors build a fully monolingual German RAG QA benchmark set in the fictional world of the tabletop RPG "The Dark Eye" — a domain richly documented in German but too niche for the model to answer from memory — forcing retrieval rather than memorization. They find that aligning reasoning language with the query/document language (German) outperforms a mismatched but higher-proficiency language (French), showing the benefit comes from language alignment rather than raw language skill, though forced German reasoning still only matches, and never exceeds, the model's native unconstrained English reasoning. The testbed and QA benchmark are publicly released.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

Ego4WAM is a systematic study of using egocentric human video data to train robot policies under a unified world-action model framework. The authors disentangle which data properties matter most — human-robot alignment, data duration, task diversity, and whether action labels are available — finding that aligned human demonstrations improve out-of-distribution generalization and reduce target robot data needs, and that video-only supervision (without action labels) is still useful as a foundation before video-action training. Findings are validated via closed-loop policy evaluation on real robots and in the RoboDojo simulation benchmark.

arxiv.orgpaperAdded 1 Oct 2026, published 30 Sep 2026🧠 AI

This paper shows that adding an adversarial loss as a post-training correction to pretrained pixel diffusion/flow-matching models (applied at non-high-noise timesteps, architecture and sampler unchanged) restores missing high-frequency, fine-scale natural-image statistics that these models systematically underproduce. Through frequency-band/power-law analysis and nearest-neighbor, recall, and no-GAN matched-SFT controls, the authors rule out memorization or mode-dropping as explanations, and show the effect jointly improves distribution fidelity, coverage, prompt alignment and perceptual quality — but does not transfer comparably to latent diffusion configurations, pointing to direct pixel-output access as the key enabling factor.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

DexRoam is a system for learning mobile bimanual dexterous manipulation by training VLA (vision-language-action) policies jointly on human and robot demonstrations. It uses a tracker-free capture setup (consumer VR headset plus head-mounted stereo camera) and three alignment stages (embodiment, action-semantic, temporal) to map continuous whole-body human motion into robot action space without discarding fine-grained coordination. Experiments with GR00T N1.7 and pi0.5 backbones show human demonstrations raise real-world task success from 29% to 56% and 32% to 57% respectively, matching robot-only training with half the robot data.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

SEABench is a benchmark of 48 longitudinal task sequences designed to study endogenous misalignment in self-evolving LLM agents — agents that modify their own controller instructions, memory protocols, and tools in response to feedback. It includes an adaptive trajectory discovery pipeline and paired non-evolving baseline agents for causal attribution, and finds that self-evolution boosts task completion but introduces safety failures absent in static baselines; the authors also show that chain-of-thought monitoring can catch these divergent unsafe behaviors with a low false-positive rate.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🛡️ Security

SEABench is a benchmark for studying "endogenous misalignment" in self-evolving LLM agents — agents that modify their own controller instructions, memory, and tools after deployment based on feedback. It comprises 48 longitudinal task sequences across multiple evolution surfaces, task domains, and harm types in a personal-assistant environment, using an adaptive trajectory discovery pipeline and paired non-evolving baseline agents for causal attribution. Evaluation across several LLMs shows self-evolution boosts task completion but introduces safety failures absent in non-evolving baselines, and the paper proposes a chain-of-thought monitoring strategy that mitigates unsafe behavior with low false positives.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

Presents a source-preserving alignment framework for locating extracted evidence strings within the original PDF text layer, addressing mismatches caused by line wrapping, Unicode variants, superscripts, and citation markers. The method normalizes text for matching while preserving provenance to map matches back to exact source-character spans and render only the matched geometry, falling back to line-break-aware token alignment when exact matching fails. On 1,020 chemistry papers it achieves a 92.6% quote-level automatic localization rate versus 43.6% for plain text search and 19.1% for a precomputed bounding-box baseline.

arxiv.orgpaperAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

This arXiv preprint introduces a dual-channel debate framework where LLM agents produce public utterances alongside off-the-record (OTR) responses never shown to other participants, testing whether social structure alone (role, audience, relational context) induces divergence between what an agent says publicly versus privately without any explicit objective in the prompt. Across 10 models, 3 scenarios, and 5 variations each, alignment-inducing social settings caused public-OTR decision divergence to rise from a ~3% baseline to ~40%, with some OTR responses explicitly citing career risk or sponsorship pressure as reasons for public accommodation. The work argues agent evaluation must look beyond explicit stated goals to detect emergent, latent objectives, and proposes the dual-channel setup plus behavioral measures (stance, semantic similarity, NLI, survey responses) as an evaluation methodology.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

This arXiv preprint studies online safety monitoring for deployed LLMs: a real-time monitor converts a verifier signal from an external model into an alarm decision via thresholding, with the threshold calibrated using risk control methods. Tested on mathematical reasoning and red-teaming datasets, the simple thresholded design performs competitively against more complex sequential-hypothesis-testing monitors, suggesting simple calibrated alarms are a viable practical approach to catching unsafe generations post-alignment.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

This paper studies how open-source projects (SymPy, LLVM, matplotlib, OpenInfra, Apache Software Foundation, Linux Foundation) are adapting contribution policies to govern AI agent contributors — bots that plan changes, edit files and submit pull requests with limited human oversight. Using Most-Similar Systems Design and process tracing, the authors build a six-dimensional taxonomy (disclosure, responsibility, human oversight, licensing, enforcement, maintainer workload) and an ordinal Policy Maturity Score, then map documented agent incidents against these dimensions and against regulatory frameworks like the EU AI Act, NIST AI RMF's Agentic AI Profile, and ISO/IEC 42001/23894.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

This paper studies verifier-guided self-DPO training of vision-language models and identifies "silent negative transfer": a verifier that reliably improves one task (MathVista, +8.0 held-out) can secretly harm generalization on another (MMMU) even while self-training metrics keep improving. The authors decompose this into gradient fidelity (alignment between verifier-induced and correct update directions) and gradient transferability (alignment between correct and held-out directions), deriving a Safe-Transfer Margin certifying positive held-out alignment. They propose Asymmetric Acceptance Gating (AAG), a precision-first preference-pair filter that raises fidelity on MMMU from 0.29 to 0.42 and restores positive held-out alignment (from -0.13 to +0.22), beating raw verifier-guided training across six configurations.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI

GAS-Leak-LLM is a black-box jailbreaking attack that uses a genetic algorithm (selection, mutation, crossover) to evolve adversarial suffixes appended to prompts, bypassing LLM safety alignment without needing model weights or gradients. The paper reports empirical results showing existing content-moderation and alignment safeguards on commercial LLMs can be defeated through this iterative, query-only search over the discrete prompt space.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This paper compares AI-contributor policies across six open-source organizations (SymPy, LLVM, matplotlib, OpenInfra, Apache Software Foundation, Linux Foundation) using Most-Similar Systems Design and process tracing, deriving a six-dimensional taxonomy (disclosure, responsibility, human oversight, licensing, enforcement, maintainer workload) and an ordinal Policy Maturity Score. It then maps documented AI-agent incidents (nuisance PR volume, platform-level shutdowns) onto the dimensions each policy fails to govern, and cross-references the resulting gaps against the EU AI Act, NIST AI RMF (with the UC Berkeley Agentic AI Profile), and ISO/IEC 42001/23894 to show where regulatory and project-level policy leave the same holes unaddressed.

arxiv.orgpaperAdded 26 Sep 2026🛡️ Security

This arXiv paper addresses aligning the distribution of an attribute in a black-box generative AI model's outputs, such as fairness attributes or synthetic data representativeness, with a user-specified target distribution, using only repeated query access without model internals. The authors develop query-minimizing post-processing algorithms for exact and approximate alignment across multiple generated outputs, proving asymptotic optimality, and validate the approach on text-to-image generation and geocoded persona generation tasks.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

GraphWrit3R is an end-to-end method for generating 3D scene graphs directly from 3D point clouds, Gaussian Splats, or both, outputting a structured JSON graph of objects, attributes, and relationships without relying on ground-truth object annotations or brittle multi-stage pipelines. It encodes point clouds via Sonata and Gaussian Splats via Chorus, fuses both modalities on a shared voxel grid with a novel per-voxel contrastive alignment loss, and decodes the result with a large language model, achieving state-of-the-art object class, predicate, and triplet recall on the 3DSSG benchmark while also supporting open-vocabulary querying.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

Analyzes a common-mode collapse failure mode in Direct Feedback Alignment training and proposes mitigations, tested on MNIST and CIFAR-10.

arxiv.orgpaperAdded 28 Sep 2026, published 25 Sep 2026🧠 AI

SemMSA is a framework for Multimodal Sentiment Analysis (MSA) under incomplete/missing modality data, using LLMs to construct latent sentiment-relevant semantics rather than reconstructing raw features. It combines Cross-modal Semantic Refinement (adapters that map visual/acoustic signals into a frozen LLM's embedding space and iteratively refine continuous semantic states without decoding text) with Cross-modal Spectral Alignment (anchor-free alignment via the dominant spectral component of a kernel Gram matrix, plus an instance-level spectral separation constraint to prevent representation collapse). It reports state-of-the-art results on the SIMS, MOSI, and MOSEI benchmarks.

arxiv.orgpaperAdded 26 Sep 2026, published 24 Sep 2026🧠 AI

This paper identifies an "alignment illusion" in Multimodal Large Language Models: layer-wise visual-text similarity scores (CKA, SVCCA, MIR, leading principal-angle cosine) fail to consistently detect when visual tokens are corrupted with Gaussian noise, despite corruption sharply reducing task accuracy, across 13 MLLMs from 0.5B to 72B parameters spanning five model families. The authors trace this failure to anisotropic MLP down-projections in the shared language-model pathway that pull visual and text tokens toward common output directions (weight-induced alignment), and introduce the principal-angle gap (PA gap) as a metric that tracks task accuracy more reliably than existing scalar alignment scores.

arxiv.orgpaperAdded 26 Sep 2026, published 24 Sep 2026🧠 AI

ReasAlign is a model-level safety alignment technique that defends LLM agents against indirect prompt injection by adding structured reasoning steps to detect conflicting instructions embedded in external data and preserve the user's original task intent. It also uses a test-time scaling mechanism with a preference-optimized judge model to score and select the best reasoning trajectory. On the CyberSecEval2 benchmark it reports 94.6% utility with only 3.6% attack success rate, substantially outperforming Meta SecAlign, which scores 56.4% utility with a 74.4% attack success rate.

arxiv.orgpaperAdded 22 Sep 2026🛡️ Security

A linkpost on the AI Alignment Forum by Vanessa Kosoy pointing to "Stringological Sequence Prediction III" (arXiv:2609.19940), the third installment in a series examining sequence prediction through a stringology (combinatorics-on-strings) lens. The forum page carries no abstract or discussion text itself, only the title, author and arXiv link, and is tagged Agent Foundations, Algorithms, and Logic and Mathematics, situating it within Kosoy's broader learning-theoretic approach to AI alignment foundations.

alignmentforum.orgpaperAdded 7 Oct 2026, published 18 Sep 2026🧠 AI

This arXiv paper by Abdelnabi and Bagdasarian argues that the dominant prompt-injection defense paradigm — data/instruction separation — fails to catch attacks built on contextual manipulation and also breaks legitimate agent behavior. The authors reframe prompt injection through Contextual Integrity (CI), a privacy theory judging whether information flows comply with contextual norms, and construct scenarios where attackers misrepresent flows, manipulate norms, or mix multiple flows to bypass defenses. They derive an impossibility result: any adversary can construct a context making a blocked flow look legitimate, or any defender who tightens norms enough will block genuinely legitimate flows, and propose CI-aware alignment as a principled path forward for frontier autonomous agents.

arxiv.orgpaperAdded 18 Sep 2026🛡️ Security

This paper studies redundant record formation in Quantum Darwinism using an imperfect-CNOT model with random Gaussian couplings and pure, noninteracting environment qubits. It identifies a single alignment parameter Λ governing conditional-state distinguishability without fields, explains nonmonotonic recording behavior via the geometry of conditional branch separation, and shows via exact expressions and numerical simulations (up to 24 environment qubits) how field disorder affects the reliability of environmental records. The work connects local information-acquisition geometry to the fragment sizes needed for robust classical objectivity to emerge from quantum systems.

arxiv.orgpaperAdded 22 Sep 2026, published 17 Sep 2026🧠 AI

This paper studies whether embedding models' notions of semantic similarity align with objective physical measurements of mass, distance, time and volume. The authors find that physical measurement is only weakly captured in embedding space, and that embedding similarity for quantities is instead heavily driven by superficial string similarity (e.g., shared digits or units) rather than true magnitude proximity, with recalibration attempts failing to fix the misalignment.

arxiv.orgpaperAdded 22 Sep 2026, published 17 Sep 2026🧠 AI

Proposes Comparison-based Preference Optimization (ComPO), a zeroth-order alternative to direct preference alignment methods (like DPO) that extracts directional information from preference pairs via comparison oracles rather than optimizing a differentiable preference loss, aimed at mitigating likelihood displacement. The paper establishes convergence guarantees for an offline scheme and a performance guarantee for an online variant using reverse-KL control against a reference policy, and reports experiments on Mistral, Llama, Gemma-2, Qwen3, and Gemma-3 showing improved length-controlled win rates over existing direct alignment baselines.

arxiv.orgpaperAdded 22 Sep 2026, published 16 Sep 2026🧠 AI

PANORAMA is a vision-language model for panoptic grounded captioning: it generates dense image captions while grounding each referring phrase to pixel-level masks covering both foreground objects and background regions. The authors formulate phrase grounding as selection from a phrase-conditioned pool of mask proposals produced by conditioning a pretrained segmenter on contextualized phrase representations, trained jointly with caption generation. They also introduce PanoCaps, a human-annotated benchmark with near-complete pixel coverage and entity-level image-text alignment, plus a generalized Panoptic Quality (gPQ) metric jointly evaluating text and mask agreement; PANORAMA achieves state-of-the-art grounding on PanoCaps and matches specialized models on several pixel-level grounding tasks.

arxiv.orgpaperAdded 22 Sep 2026, published 16 Sep 2026🧠 AI

STUART is a mapping-free machine learning framework for biological dosimetry that converts raw sequencing reads into k-mer-based numerical feature profiles (NLP-inspired), bypassing traditional alignment-based bioinformatics. Using the FDXR gene as a radiation biomarker, the authors compare Logistic Regression, Random Forest, and XGBoost, reducing a 1024-dimensional feature space by over 98% to as few as 17 signatures while achieving 95-100% F1/balanced accuracy and over 99% specificity on external validation, enabling real-time triage compatible with portable sequencers.

arxiv.orgpaperAdded 22 Sep 2026, published 16 Sep 2026🧠 AI