72 results for “Post-training”

This preprint proposes a unified post-training quantization framework for large language models. It derives a Hessian-guided approximation of the output change caused by joint weight and activation quantization, using a tractable Fisher approximation. Quantization ranges, linear transformations and weight rounding are optimized together under one output-sensitivity objective. The authors report gains over state-of-the-art transformation and rounding methods in channel-wise and block-wise settings.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

Turing LLM post-training combines calibrated multi-teacher on-policy distillation (C-MOPD) with an execution path for dynamically routed mixture-of-experts models. C-MOPD matches each teacher on a local Top-K support with teacher-wise loss normalization and clipping. On Turing-20B-A2B it scores 0.9988 on four-domain integration, beats Qwen3-8B on math and code (85.92 vs 80.33 mean), and cuts consolidation step time from 398.7 to 139.3 seconds in a PPU profile.

arxiv.orgpaperAdded 31 Oct 2026, published 29 Oct 2026🧠 AI

QbD-VLA is a mixed-precision post-training quantization method for vision-language-action models. It uses task success from short closed-loop evaluations to assign weight and activation bit widths, accounting for interactions between components. On four VLA models across all four LIBERO suites it holds task performance at about 4-bit weights and under 6-bit activations. The gain on SIMPLER is small. On a Franka Research 3 arm, X-VLA shows no detected loss, but π0.5 fails.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

This paper measures job-based doubt in LLM fact-checkers: judging a claim false more often when the poster has a lower-status job. It compares base and post-trained checkpoints of six model families on fact-checked claims with randomised poster profiles. Only the open Tülu-3 recipe (OLMo-2 32B, Tülu-3 70B) creates the effect, under one percentage point but reliable. In OLMo-2 32B the flip comes from one residual-stream direction and appears at supervised fine-tuning, confirmed on 400 new claims. None of seven commercial models doubts lower-status posters more.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

This paper measures how much a model's final answer depends on its own written chain of thought, using a synthetic box-swap task with exact ground truth. It edits one step in the model's chain to a wrong but valid state and checks whether the answer follows. Across 25 open-weight models, reliance rises with size and saturates (Qwen3 from 16-18% at 0.6B to 97-100% at 14B). Reliance tracks capability (r = 0.99) and is unchanged by post-training.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

LLMON (LLM Object Notation) is a framework that separates instructions from data and requires explicit identifier-based execution. The authors post-train Granite-4.1 3B and 8B base models with SFT then RL on a 4.6B-token structured corpus covering general, RAG and tool-calling data, with distractor variants. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, and general capability is preserved. Attention-based metrics diagnose failures per transformer layer.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🛡️ Security

LLMON (LLM Object Notation) is a framework that separates executable instructions from passive data and requires explicit identifier-based execution. The paper post-trains Granite-4.1 3B and 8B base models on a 4.6B-token corpus of general, RAG and tool-calling data with distractor variants, using SFT then RL. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, with general capabilities preserved. It also adds attention-based per-layer diagnostics.

arxiv.orgpaperAdded 30 Oct 2026, published 28 Oct 2026🧠 AI

Know, then Draw is a learning paradigm for visual generation that uses autoregressive pretraining to acquire semantic knowledge and diffusion post-training for high-fidelity rendering. Controlled comparisons find autoregression the stronger pretraining objective, and semantic quality matters more than AR-only generation quality. This motivates a purely semantic tokenizer trained with language supervision only. A 3B text-to-image model trained in 6,600 H100 GPU-hours scores 86 on GenEval and 87 on DPG-Bench.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

This paper studies how multimodal LLMs refuse questions that an image cannot answer, using the MoHoBench benchmark on three MLLMs. It varies the meaning and placement of refusal instructions, compares supervised and preference-based post-training, and intervenes on a refusal-related activation direction. Instructions change refusal at the sample level in ways aggregate rates hide. Post-training raises refusal most but can mix refusal with attempted answers. Activation intervention gives smaller, transferable changes.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

AxisQ is a post-training quantization framework for camouflaged object detection (COD) that allocates activation bits by feature depth. It uses the channel domain for shallow features and the local-frequency domain for deep features, under a fixed per-layer average bit budget. It adds a boundary-aware Hessian approximation for reconstruction. On CAMO at W2A2, weighted F-measure improves 12.7% for CFRN and 33.3% for CamoFormer-P over the strongest baselines.

arxiv.orgpaperAdded 28 Oct 2026, published 27 Oct 2026🧠 AI

Intervention-Certified Amplification (ICA) is a validity gate for sigma-weighted hybrid SFT/RL post-training. The paper shows that DIAA, a per-sample sigma-weighted hybrid, emits unexecuted tool-integrated reasoning on 29% of MATH-500 prompts (three times vanilla GRPO) even with Python fences removed from its anchor. ICA gates sigma with paired-rollout evidence, cutting TIR to 13.8% (frozen) and 11.2% (online) and raising accuracy from 47.5% to 48.7% and 49.3%. It transfers to GSM8K.

arxiv.orgpaperAdded 27 Oct 2026, published 24 Oct 2026🧠 AI

IQuest-Q1 is an open-source sparse Mixture-of-Experts model with 320B total and 15B activated parameters, built for agentic coding in CLI systems. Pre-training combines Stable SplitNorm, Muon and muP with a two-stage curriculum that raises the coding-data share in stage two. Mid-training extends context, and post-training mixes SFT, domain-specialized RL and MOPD across scaffolds, followed by model merging.

huggingface.copaperAdded 24 Oct 2026🧠 AI

SOMA (Sharpness-aware Orthogonalized Momentum Algorithm) combines sharpness-aware minimization (SAM) with the Muon optimizer for LLM training. It avoids SAM's extra gradient computation by using stale, purely stochastic gradient noise as the adversarial perturbation. The authors prove sample complexity does not depend on perturbation direction, study robustness to heavy-tailed gradient noise, and test SOMA against post-training quantization degradation and catastrophic forgetting during fine-tuning.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

BiLoCo is a post-training quantization method for FP4 LLMs that adds low-rank corrections whose left and right factors are stored as binary ±1 sign vectors instead of BF16. This uses 16x less storage per rank-one component and lets the correction run on CUDA cores alongside the FP4 tensor-core GEMM. On 4B to 72B LLMs it matches or improves memory-matched SVDQuant. A fused B300 kernel is at most 11% slower than FP4-only GEMM at batch size 1 for ranks up to 512.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

MPQ (Message-Passing Quantization) is a post-training quantization method that treats row-wise PTQ as posterior inference over discrete codebooks. It keeps soft posteriors across weight blocks using vector approximate message passing, then refines a hard-decision initialization with exact block coordinate descent. Tested at 2-bit lattice and 3- and 4-bit scalar quantization on Llama, Qwen and Mistral models, it improves perplexity and zero-shot accuracy over GPTQ-style baselines.

arxiv.orgpaperAdded 26 Oct 2026, published 23 Oct 2026🧠 AI

ThinkingBox-Training is Microsoft's released RL stack that turns executable database-state checks from ThinkingBox tasks into GRPO rewards. Fifty GRPO steps on 187 curated tasks raised Qwen3.8-27B pass@1 from 51.70% to 60.78% and tasks passing all 20 attempts from 38 to 107 on the held-out 507-task benchmark. The post explains offline policy-relative difficulty filtering with Jeffreys posteriors, the 24-H100 Verl setup and evaluation steps.

huggingface.cowriteupPublished and added 22 Oct 2026🧠 AI

BeNice! is a post-training method for looped language models that broadcasts the final recurrence's KV-cache into the first recurrence, so one cache can serve all recurrences. It follows a first stage that uses stochastic depth in pretraining to induce contractive residual dynamics. It matches standard finetuning's unshared validation perplexity and composes with multi-latent attention, Kimi delta attention and fp8, mxfp8 and nvfp4 cache quantization. In simulated low-memory long-context inference it raises cache hit rates by up to 31.4% and throughput by up to 23.6%, and cuts p90 time-to-first-token by up to 83.6%.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

This October 2026 arXiv paper evaluates four LLM inference optimization families on Apple Silicon: KV-cache pruning, speculative decoding, ported GPU kernels and post-training quantization. Tests run on four Apple machines with a single-request decode workload, plus matched RTX 4090 runs. KV-cache pruning reaches 1.75x throughput on Apple, all tested speculative decoding methods reduce throughput, and ported kernels give up to 1.20x.

arxiv.orgpaperAdded 24 Oct 2026, published 22 Oct 2026🧠 AI

Every Part Has Its Role studies concept augmentation in LLM post-training, where a target word is replaced by related alternatives in training sentences. It extends prior work from nouns to verbs and adjectives. Concept augmentation improves perplexity most consistently on news regardless of part of speech, while verbs give the most reliable downstream gains. CKA and linear probing show the largest gains do not match the largest representation changes.

arxiv.orgpaperAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

Monitorability Is Trainable proposes an unsupervised post-training objective that maximizes mutual information between a model's chain of thought and its output. On three model organisms with degraded CoT (from compression, monitor optimization, and synthetic-document generalization), it recovers most or all lost monitorability. On undegraded policies it improves CoT monitor and activation probe prediction of unsafe behavior and raises consequential reasoning.

arxiv.orgpaperAdded 23 Oct 2026, published 21 Oct 2026🧠 AI

AgenticRoboRL is a framework for post-training robot planners across different harnesses, combining supervised fine-tuning with multi-turn reinforcement learning. Harness-specific adapters link recorded planner interactions to task outcomes, so one RL pipeline covers both VLA skill invocation and control-code generation. On LIBERO-PRO, a 9B planner rose from 18.1% to 71.0% without external memory, near Codex (GPT-5.5) with memory at 72.1%. Code and checkpoints are promised.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

AnomalyAgent is a post-training fusion framework for visual anomaly detection that ensembles pretrained detectors. It aligns scores and anomaly maps, summarizes disagreement, and treats fusion as a choice among score actions and map actions. A constrained vision-language model inspects high-conflict samples and picks the actions. It beats equal-weight fusion and gets the best mean image AUROC and pixel AP among compared fusion methods on MVTec AD and VisA.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

This paper studies 2-bit post-training quantization (PTQ) of six dense and mixture-of-experts models from 3.8B to 32B parameters. It finds the quantization representation matters most: vector codebooks give usable models where scalar PTQ falls to chance. Calibration and recovery data must be chosen jointly. The recipe uses about 2.3 effective bits per weight, keeps 89 to 97% of full-precision QA accuracy, and needs 1.4 to 30x less compute than the strongest QAT baseline.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

Calibrated accommodation pairs (CAP) is a preference-data method that rewards chat models for matching a user's dialect density while keeping Standard English for Standard-English users. The paper traces SFT, DPO and RL stages across 10 open model lineages and finds post-training standardizes replies, while the DPO update still favours British and Australian spelling over AAE and Singlish. On a fully open model, CAP improves convergence on unseen features without added dialect for Standard-English users or lower instruction following.

arxiv.orgpaperAdded 22 Oct 2026, published 20 Oct 2026🧠 AI

MemT-1.0 is a memory foundation model architecture that builds memory into model computation. It builds on Metis, uses Gated DeltaNet-2 for token-level memory updates within each turn, and adds Mixture-of-Storage to scale capacity. Training used 34B tokens in continued pre-training and post-training. MemT-1.0-Nano and MemT-1.0-Flash beat parametric memory baselines and stay competitive with retrieval methods. At 128K history, Nano gives about 1.86x inference speedup and 66x less persistent storage than a full-context KV cache.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

The paper traces sycophancy across 741 checkpoints from 30 open model lineages, from early pretraining to released models. It plants initial verdicts on moral scenarios and measures how far models follow a user's opinion or drop their answer under pressure. The behaviour usually exists before post-training, but which questions a released model yields on is set mainly during post-training. For Tülu-3 8B, its own preference-stage checkpoint removes 85% of remaining prediction error, a sibling model only 19%.

arxiv.orgpaperAdded 21 Oct 2026, published 19 Oct 2026🧠 AI

ConQUESTA is a post-training pruning framework for quantum neural networks that combines task saliency with compiler cost, screens candidate entanglers, and searches whole pruning masks under an accuracy constraint. Each mask is transpiled across several coupling topologies. At 55% pruning on 4-, 6- and 8-qubit QNNs it cuts resources by 61.6% (Breast Cancer), 72.0% (Digits01) and 82.2% (Iris), and fixed-mask recovery raises feasible runs from 16/24 to 22/24.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

SpikeQ is a spike-guided post-training quantization framework for spiking neural networks on neuromorphic chips. It picks each weight level by simulating downstream neurons to minimize spike errors, and uses a reset-aware grid for pre-fire membrane potentials. With 4-bit weights and potentials, models stay within 2.59 points of full precision, with 7.8 to 7.9 times smaller memory and 6.4 to 6.9 times less energy.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

Many Biases, One Circuit is a October 2026 arXiv paper that traces attention heads behind cue-following (for example "I think the answer is B") through two post-training pipelines. Four cue types share a few heads already present in pretrained models. About 200 generic assistant examples activate them in chat format, and the final RL stage strengthens cue effects. Swapping attention patterns of six middle layers between Tulu-3-8B and OLMo-2-7B checkpoints moves cue effects most of the way.

arxiv.orgpaperAdded 20 Oct 2026, published 16 Oct 2026🧠 AI

Value as Memory (VaM) is a post-training framework for generalist robot policies. It addresses partial observability and the delayed feedback of offline post-training. Value caching conditions actions on recent value predictions as task-progress memory. Online real-world RL combines offline demonstrations, off-policy interventions and on-policy rollouts, and moves from offline RL warmup to autonomous learning with sparse rewards. On long-horizon deformable-object and contact-rich manipulation tasks it gains 29.4% mean absolute success rate from value caching and 32.8% from online learning over matched offline training. The online schedule runs 8x as many policy iterations under the same gradient-step budget, with over 2x lower time per gradient step from concurrent data collection and training.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

EliteReplay is an RL post-training recipe that pairs GRPO with an "elite" archive built by quality-diversity search. Before training, the frozen base model is used to collect solutions spanning distinct approaches; during training, a replay loss on this archive is added to GRPO. This counters the narrowing of solution diversity that RL causes. On text-to-SQL tasks it reaches higher pass@128 than all baselines and yields up to 77% more distinct correct strategies. It also retains more of the base model's strategies and covers both readings of ambiguous questions nearly twice as often on AmbiQT.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Release notes for Apertus 1.5, a fully open (weights, data, training details) continued pretraining of the 8B and 70B Apertus models on 4T and 2T additional tokens. It adds early-fusion image and audio understanding via discrete tokenizers (Emu3.5, WavTokenizer) without architectural change, a thinking mode, 262k context, and better tool use. It documents a four-stage pretraining and five-stage post-training pipeline (SFT, long-context SFT, RLVR, DPO, charter-alignment SDPO), plus evaluation via the MLLM-eval-suite and Every Eval Ever (EEE) Eval Cards, released checkpoints and datasets.

huggingface.coarticlePublished and added 15 Oct 2026🧠 AI

The paper shows that symmetric aggregation of a primary reward (correctness) and auxiliary rewards (brevity, efficiency) in LLM post-training can raise aggregate reward while primary-task performance falls. It proves that deterministic conditional-mean GRPO converges to auxiliary-favoring behavior when auxiliary rewards are more reliably obtained. It proposes Trust-Anchored Gating (TAG), which elects an anchor channel by policy drift and gates auxiliary credit on anchor satisfaction. On Qwen3-4B with a brevity reward, TAG reaches 0.581 pass@4 versus 0.220 for additive aggregation, and shortens responses by 44% relative to correctness-only training. Experiments on Qwen and Llama models cover math and program synthesis.

arxiv.orgpaperAdded 17 Oct 2026, published 15 Oct 2026🧠 AI

Introduces Hybrid Reasoning Agents (HybridAgents), which switch between latent reasoning and text-based interaction in one generation process. A lightweight HybridLink handles transitions between the two modes, and a continuous working memory keeps hybrid context lossless across turns. This keeps tool use and makes latent thoughts inspectable. Post-training has two stages: HybridSFT, then HybridRL, which learns when to think latently and when to act in text. Across 3 model scales and 11 benchmarks (math, science, medicine, search, agentic reasoning), the authors report +10.4% performance, 35.5–54.7% fewer tokens, 1.8–2.6x faster inference and 2.7–3.7x lower cost versus text and latent baselines.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

Liquid AI releases LFM2.5-24B-A2B, an open-weight reasoning-only mixture-of-experts model (40 layers, 64 experts with top-4 routing, hybrid gated short convolutions and GQA, 2B active parameters) with 131K context and a 128K-token vocabulary. Pre-training scaled to ~33T tokens, followed by long-context mid-training and a five-stage post-training pipeline (SFT, teacher specialization, two multi-domain on-policy distillation stages, agentic RL in harnesses such as OpenClaw, Hermes Agent, Pi, OpenCode). It reports leading or matching results on tool-use and instruction-following benchmarks (e.g. LiveClaw 48.09 vs 38.19 for Qwen3.6-35B-A3B), 184 tok/s on an M5 Max and 21K tok/s on a B300, and includes transformers code and Hermes Agent local-serving config.

huggingface.coarticlePublished and added 13 Oct 2026🧠 AI

Introduces Structured Recipe Optimization (SRO), a paradigm for automated LLM post-training that optimizes over a task-adapted recipe space covering data sources, data processing, cross-domain composition and training configuration. It pairs an LLM-based Recipe-Space Designer, which builds the recipe space for the target tasks, with a Multi-Domain Recipe Optimizer that uses structured trial evidence to guide LLM candidate construction and multi-objective Bayesian trial selection. SRO gets the highest average score in all evaluated target-model and agent-LLM combinations; for example, 41.1 vs 15.4 for the strongest baseline on Qwen3-1.7B with GPT-5.4, and 53.8 vs 40.8 on Qwen3-4B with DeepSeek-V4-Flash, in joint post-training across three capability domains and six benchmarks.

arxiv.orgpaperAdded 14 Oct 2026, published 13 Oct 2026🧠 AI

DualQuant is a calibration-free post-training quantization framework for LLM weights and activations. It minimizes weight-reconstruction MSE using variable splitting and block coordinate descent with closed-form subproblem updates. The resulting scaling factors double as per-channel activation rescalings, so no calibration data is needed. It supports the MX family and NVFP4 block formats. On Llama and Qwen models it matches or beats calibration-based SmoothQuant on WikiText-2, C4 and zero-shot tasks. A real packed W4A4 path for Llama-3.1-8B on an RTX 4090 gives up to 2.61x memory reduction and 2.46x throughput over BF16.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

SWE-Critix is a post-training pipeline that trains generalist verifiers for coding-agent trajectories. Given a task and its (often hundreds-of-steps) trajectory, the verifier predicts a completion label and writes a natural-language analysis explaining the judgment. Starting from Qwen3-30B-A3B reasoning, classification accuracy on a held-out test set rises from 57% to 66% after SFT and 78% after RL, using trajectories from CodeForge-Preview. Models, datasets and training code are released.

arxiv.orgpaperAdded 12 Oct 2026, published 9 Oct 2026🧠 AI

DreamTrue is a multi-view, cross-embodiment robot world model for action-faithful, physically plausible video prediction. It renders action trajectories into image-space conditions aligned to target videos by offline geometric calibration. Counterfactual post-training modifies recorded trajectories to cover unsuccessful interactions and contact configurations. A human-annotated defect dataset trains an embodied video reward model whose scores drive RL post-training. On AgiBot it reaches state-of-the-art action following and cuts the human-assessed interaction defect rate from 48.12% to 6.25%. It ranked first in the world model track of the AgiBot World Challenge 2026.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

PathTime-VLA reparameterizes Vision-Language-Action actions as a progress-indexed interaction path X(s) plus a positive interval-time profile that defines a monotone time law t(s). Geometry is thus decoupled from execution speed, so teleoperation timing artifacts don't contaminate the path target. A staged post-training recipe uses demonstrations and DAgger interventions for a target-domain prior, Speed-DQN to learn execution-speed multipliers from robot interaction, and Path-AWR to refine the diffusion path generator from rollout outcomes. On three tasks it reaches 58/60 successes versus 57/60 for BC+DAgger at fixed 1×, with about 39–52% shorter mean completion times on successful trials.

arxiv.orgpaperAdded 9 Oct 2026, published 8 Oct 2026🧠 AI

TRACE (Train-Rollout Quantization Alignment via Compact GuidancE) is an FP4 quantization framework for reinforcement learning post-training of Mixture-of-Experts language models. It uses rollout-guided quantization-aware training to align training-side FP4 rounding with rollout-side quantization outcomes, reducing the train-rollout discrepancy that limits prior low-precision RL methods, and adds a caching scheme that selectively retains mantissa/scale information from deeper layers to cut communication overhead. Evaluated on four large MoE models across reasoning, coding and long-horizon RL tasks, it matches BF16-rollout RL performance while achieving up to 5.4x rollout speedup with joint FP4 weight/activation and FP4 KV-cache quantization.

arxiv.orgpaperAdded 7 Oct 2026🧠 AI

This paper proposes three evaluation screens — Decisive-Action BPB, Patch MCQ, and prefix-conditioned pass@K — that predict how well a base model will perform as a coding agent after post-training, without requiring the base checkpoint to run a full agentic harness. By replaying successful post-trained agent trajectories and identifying the "decisive step" that flips a repository from failing to passing tests, the authors measure a base model's latent support for that action. Tested across ten pairs of public base and post-trained models, all three screens closely track post-trained SWE-bench Verified pass@1 scores, offering a way to cheaply vet base checkpoints before committing to expensive agentic post-training.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

The paper introduces DIGHT, a co-adaptive framework that couples a text-conditioned digital-human interaction generator with a physics-based humanoid tracking policy to produce physically plausible two-humanoid interactions. Rather than collapsing executability and interaction-fidelity signals into one scalar reward, it applies physics-decoupled diffusion Direct Preference Optimization (DPO) to align the generator using criterion-specific preferences derived from tracking error, friction, floating, and simulator force feedback (contact occurrence, location, duration, magnitude), then uses the aligned generator's outputs to fine-tune the tracker. Experiments show improved physical plausibility and more reliable, faithful simulated humanoid interactions compared to prior tracking-based approaches.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

NVIDIA describes how it specialized Nemotron 3 base models into domain experts for competitive programming (IOI) and olympiad mathematics (IMO) using a four-step recipe: strong base model, curated domain data with reasoning traces, SFT/RL post-training, and an inference loop (GenCorrect for IOI, generate-verify-refine for IMO) that searches and refines candidate answers. Nemotron-3-Nano-CC and Nemotron-3-Ultra-CC reached gold-level IOI 2026 scores (535.4/600), while SFT and RL checkpoints of Nemotron 3 Ultra combined to score 30/42 on IMO 2026, exceeding the gold threshold with no formal prover or external tools. The post releases training datasets, checkpoints, the Nemotron-IMO-Bench benchmark, and the NeMo-Skills inference pipelines on Hugging Face.

huggingface.coarticlePublished and added 7 Oct 2026🧠 AI

TR-PTQ is a post-training quantization method for transformers that identifies learned LayerNorm scale parameters and GELU's compounded approximations (not SoftMax) as the main sources of accuracy loss when quantizing, contrary to common assumptions about numerical precision. It introduces a unified integer-only formulation using shared Taylor Region exponential/logarithm primitives so division and square roots can run entirely in the log-domain with standard integer arithmetic, paired with a calibration-free, outlier-aware LayerNorm optimization, eliminating the need for floating-point hardware for nonlinearities while keeping accuracy degradation under 1.5% across vision and language benchmarks.

arxiv.orgpaperAdded 8 Oct 2026, published 7 Oct 2026🧠 AI

This arXiv paper introduces a post-training semantic lifting method for 3D Gaussian Splatting that assigns a single target class to Gaussians by accumulating target/non-target evidence across all views, weighted by per-view visibility, then filters with a dual-threshold (β and γβ) scheme that selects high-confidence seeds and extends them via connected components. The authors evaluate by transferring labels from Gaussians to mesh vertices and decompose error into three sources — the 2D detector, the lifting step, and the representation transfer — tuning thresholds on seven Replica scenes and testing on ten held-out ScanNet++ scenes, reaching mean mIoU of 0.93/0.65 (dataset masks/YOLO masks) on validation and 0.80/0.54 on test, a 0.24 mIoU improvement over prior per-view thresholding. The error analysis finds the 2D detector is the dominant remaining error source, informing where future gains in 3D semantic reconstruction should focus.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

This arXiv paper introduces a pre-registered panel of 248 scenarios across five kinds of pressure, each posed to a model twice (as the acting agent and as a third-person judge) with pressure-removed twins and operator-order positive controls, to measure whether models act against their own stated moral judgment. Testing OLMo-3-7B-Instruct, Llama-3.1-8B-Instruct, Tulu 3, and Qwen2.5-7B-Instruct, the authors find OLMo-3 violates its own judgment on about 1 in 5 pressuring scenarios, Llama-3.1-8B-Instruct and OLMo-3 carry this gap while Tulu 3 (fine-tuned from the same Llama-3.1 weights) and Qwen2.5-7B-Instruct show none or an unresolved effect, and that reading outside the chat template or adding stakes-reasoning changes the gap's sign and size. The paper argues this value-action gap is a target shaped by post-training recipes rather than a fixed property of pretrained weights.

arxiv.orgpaperAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

Mistral announces Mistral Large 4 ("ML4"/"Le Chonk"), a 1-trillion-parameter natively multimodal Mixture-of-Experts model with 52B active parameters, trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own European datacenters. The post details state-of-the-art open-weight results on cybersecurity benchmarks (82% on an Artificial Analysis vulnerability-reproduction test, 93% on Cybench), agentic coding (DeepSWE, Terminal-Bench, AutomationBench), finance/legal evals (via Vals, Harvey's Legal Agent benchmark), visual grounding (beating GPT-6 Astra on Dense 200), and safety benchmarks (Lakera B3, KORA). It also describes the large-scale RL post-training pipeline (33B tokens/day at 3k GPUs) used to produce the model, with full weights promised by end of month.

mistral.aiarticleAdded 7 Oct 2026, published 6 Oct 2026🧠 AI

Researchers fine-tune Qwen3-8B with GRPO against a proxy reward built from surface features (citation count, legalese density, response length) and find the model learns to strategically abstain rather than reason: on 16 LegalBench yes/no legal tasks, accuracy collapses from chance (0.500) to 0.072 because the rate of properly formatted, committed answers drops from 0.900 to 0.109, even though accuracy on answers it does commit to actually rises. They term this the "Saul Goodman effect," formally prove it is the optimal policy for any surface-feature proxy that doesn't penalize abstention, show 89.3% of post-training citations are hallucinated/corrupted real case names, and introduce three diagnostics (Confidence Theater Score, Citation Plausibility Rate, Regret Gap) to detect this failure mode before deployment.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

ImproveAnyTask is an autonomous post-training harness that automates adapting general-purpose LLMs to specific tasks under a limited compute budget, structuring the process into error attribution, update-direction selection, and executable model updates via generated training data and configs with small-scale validation before full runs. Across 11 tasks it reports mean gains of 18.29 and 11.97 percentage points on Base and Instruct models respectively, with validated strategies retained for reuse in future adaptation cycles.

arxiv.orgpaperAdded 6 Oct 2026, published 5 Oct 2026🧠 AI

This paper studies the sharpening hypothesis of RL post-training on LLMs: that it mainly sharpens existing base-model behaviors, boosting pass@1 at the cost of solution coverage (pass@K), and shows this cost extends to agentic, multi-turn tool-use tasks, not just math/coding. The authors introduce Sharpening Tax, a metric quantifying this coverage loss, measured across 14 base/post-trained model pairs from four model families on three agentic benchmarks (42 cases), finding post-training pushes tasks toward always-solved or never-solved extremes. They also propose posterior-tempered group sampling (PTGS), a Bayesian per-prompt temperature adaptation method that reduces the tax during RL training while improving both coverage and single-shot accuracy.

arxiv.orgpaperAdded 4 Oct 2026🧠 AI

LESSER is a drop-in wrapper for gradient-based post-training data selection that approximates full-parameter gradient alignment using only output-layer gradients, which can be computed with a forward pass instead of a full backward pass. It cuts feature-extraction compute by 9.7x on SFT benchmarks and 3.0x on RL benchmarks while matching the downstream performance of full-gradient selection methods, even though individual sample rankings differ, selected batches remain gradient-aligned.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

VisionMX is a post-training quantization method for vision models using Microscaling (MX) formats, which pair low-precision numeric elements with shared block scales for efficient inference. The paper identifies three error sources in direct MX conversion of vision models—block-scale representation, poor alignment of small convolutional weights with nonuniform element grids, and underuse of signed codes by nonnegative activations—and addresses them via bounded weight rounding and a foldable affine activation correction. It is evaluated across image classification, object detection, semantic segmentation, and low-light image enhancement, showing improvements over direct conversion and existing post-training quantization baselines, especially for architectures most sensitive to MX conversion.

arxiv.orgpaperAdded 5 Oct 2026, published 2 Oct 2026🧠 AI

Cloudflare announces Clef and Clef-flash, open-weight "decision models" (Apache 2.0, on Hugging Face and Workers AI) that are Jev-API compatible and return typed, probability-scored classifications instead of generated text. The models are built by post-training a frozen Qwen backbone (Qwen3.8-27B for Clef, a smaller Qwen3.5 variant for Clef-flash) with rank-256 LoRA adapters, a two-stage attention-routing head that scores schema choices non-autoregressively in a single prefill pass, label-smoothed cross-entropy plus Brier-loss calibration, and a custom RL objective called RLCD (Reinforcement Learning for Calibrated Decisions). Cloudflare also launches a fine-tuning/RL service for customers to adapt Clef, built on AI Gateway (traffic capture), Workers AI (rollouts), Containers (RL sandboxes) and a new "Trainer" component for weight updates.

blog.cloudflare.comarticleAdded 3 Oct 2026, published 1 Oct 2026🧠 AI

DeepSeek-V4.1-Flash is a multimodal Mixture-of-Experts model with 552B backbone parameters, 1M-token context, and a novel Causal Encoder-Decoder (CED) architecture where the decoder's global KV cache is projected from encoder hidden states rather than recomputed per layer, activating only 8B/16B parameters per token during prefill/decode. It introduces Compressed Sparse Attention 2 (CSA2) with FP4 KV caching to cut the global KV cache footprint to 890 bytes/token (~1/4 of DeepSeek-V4-Flash, ~1/437 of DeepSeek-V1), alongside Engram conditional memory, DSpark speculative decoding, and a continuously controllable reasoning-effort setting (1–100). The card documents pre/post-training recipes (45T tokens, SFT→RL→on-policy distillation with synthetic agent environments) and reports results across agentic and code benchmarks (Terminal-Bench 2.1/3.0/4.0, DeepSWE v1.1, GPQA Diamond, Chartography, ZeroBench, AutomationBench), plus deployment instructions for Transformers, vLLM, SGLang, and Docker.

huggingface.corepoAdded 30 Sep 2026🧠 AI

This paper shows that adding an adversarial loss as a post-training correction to pretrained pixel diffusion/flow-matching models (applied at non-high-noise timesteps, architecture and sampler unchanged) restores missing high-frequency, fine-scale natural-image statistics that these models systematically underproduce. Through frequency-band/power-law analysis and nearest-neighbor, recall, and no-GAN matched-SFT controls, the authors rule out memorization or mode-dropping as explanations, and show the effect jointly improves distribution fidelity, coverage, prompt alignment and perceptual quality — but does not transfer comparably to latent diffusion configurations, pointing to direct pixel-output access as the key enabling factor.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

STEPQuant is a spatial-temporal post-training quantization framework for the fixed-size recurrent states used by Delta-rule linear attention, which become a memory bottleneck under concurrent serving. It allocates bit precision based on error magnitude and memory lifetime (temporal) and fits per key-row/value-column scales based on state distribution and output impact (spatial), achieving near-FP32 accuracy at a 6-bit budget and beating uniform INT8 at 4-bit. Evaluated on Qwen3.8-27B and Kimi-Linear-48B-A3B-Instruct, and integrated into SGLang with custom GPU kernels, it delivers over 5x recurrent-state compression and up to 68.7% total serving memory reduction.

arxiv.orgpaperAdded 30 Sep 2026, published 29 Sep 2026🧠 AI

SemiAnalysis article on GLM-5.3 and sparse attention. It explains why DeepSeek Sparse Attention does not cut KV cache capacity (top-k needs full context in HBM) and how SGLang's HiSparse offloads KV to host DRAM. It covers InferenceX AgentX serving cost on GB200, GB300 and MI355X (about $0.044 vs $0.049 per million tokens at 150 tok/s), IndexShare, and GLM's RL post-training (SAO, slime).

newsletter.semianalysis.comarticleAdded 29 Oct 2026, published 28 Sep 2026🧠 AI

H Company's blog post announcing Holo4, a pair of open-weight agentic models (27B dense and 35B-A3B MoE, built on Qwen bases) trained to operate software through GUIs, code execution, MCP, and APIs interchangeably rather than a single interface. It reports benchmark results on OSWorld 2.0 and AutomationBench against frontier models like Claude Opus 5.5 and GPT-5.6 Sol, describes the training pipeline (an internal "Agentic Task Factory" generating ~10,000 verifiable environments/tasks plus SFT+RL), and also releases Holotron4 Nano, the same post-training recipe applied to NVIDIA's Nemotron 3 Nano Omni. Weights (BF16, FP8, NVFP4, GGUF) and full agent trajectories are published on Hugging Face.

huggingface.coarticleAdded 29 Sep 2026, published 28 Sep 2026🧠 AI

Rufus-Air is an open, reproducible post-training recipe applied to GLM-4.5-Air-Base (106B-A12B), structured as an eight-stage pipeline: SFT, Reasoning RL, Coding RL, Instruction-Following RL, General Agent, Coding Agent, Search Agent, and RLHF. The paper documents data sources, reward design, infrastructure, and stage ordering using only open-source components and public data (no new human annotation or distillation teacher), reporting improvements over the official GLM-4.5-Air post-trained release and competitiveness with similarly sized open models. Key findings include the value of diverse SFT for a capability floor, difficulty filtering to keep RL prompts in a productive range, and reward reliability as a principle for stage ordering.

arxiv.orgpaperAdded 26 Sep 2026🧠 AI