Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei
18515
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling
paradigm for large language models and reinforcement learning (RL).
Specifically, we reframe next-token prediction as a reasoning task trained
using RL, where it receives verifiable rewards for correctly predicting the
next token for a given context. RPT offers a scalable method to leverage vast
amounts of text data for general-purpose RL, rather than relying on
domain-specific annotated answers. By incentivizing the capability of
next-token reasoning, RPT significantly improves the language modeling accuracy
of predicting the next tokens. Moreover, RPT provides a strong pre-trained
foundation for further reinforcement fine-tuning. The scaling curves show that
increased training compute consistently improves the next-token prediction
accuracy. The results position RPT as an effective and promising scaling
paradigm to advance language model pre-training.
LASA Team, Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, Yu Rong
933
Multimodal Large Language Models (MLLMs) have demonstrated impressive
capabilities in understanding common visual elements, largely due to their
large-scale datasets and advanced training strategies. However, their
effectiveness in medical applications remains limited due to the inherent
discrepancies between data and tasks in medical scenarios and those in the
general domain. Concretely, existing medical MLLMs face the following critical
limitations: (1) limited coverage of medical knowledge beyond imaging, (2)
heightened susceptibility to hallucinations due to suboptimal data curation
processes, (3) lack of reasoning capabilities tailored for complex medical
scenarios. To address these challenges, we first propose a comprehensive data
curation procedure that (1) efficiently acquires rich medical knowledge data
not only from medical imaging but also from extensive medical texts and
general-domain data; and (2) synthesizes accurate medical captions, visual
question answering (VQA), and reasoning samples. As a result, we build a
multimodal dataset enriched with extensive medical knowledge. Building on the
curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu
undergoes multi-stage training to embed medical expertise and enhance its
task-solving capabilities progressively. Besides, we preliminarily explore the
potential of applying reinforcement learning with verifiable rewards paradigm
to enhance Lingshu's medical reasoning ability. Additionally, we develop
MedEvalKit, a unified evaluation framework that consolidates leading multimodal
and textual medical benchmarks for standardized, fair, and efficient model
assessment. We evaluate the performance of Lingshu on three fundamental medical
tasks, multimodal QA, text-based QA, and medical report generation. The results
show that Lingshu consistently outperforms the existing open-source multimodal
models on most tasks ...
This paper introduces MiniCPM4, a highly efficient large language model (LLM)
designed explicitly for end-side devices. We achieve this efficiency through
systematic innovation in four key dimensions: model architecture, training
data, training algorithms, and inference systems. Specifically, in terms of
model architecture, we propose InfLLM v2, a trainable sparse attention
mechanism that accelerates both prefilling and decoding phases for long-context
processing. Regarding training data, we propose UltraClean, an efficient and
accurate pre-training data filtering and generation strategy, and UltraChat v2,
a comprehensive supervised fine-tuning dataset. These datasets enable
satisfactory model performance to be achieved using just 8 trillion training
tokens. Regarding training algorithms, we propose ModelTunnel v2 for efficient
pre-training strategy search, and improve existing post-training methods by
introducing chunk-wise rollout for load-balanced reinforcement learning and
data-efficient tenary LLM, BitCPM. Regarding inference systems, we propose
CPM.cu that integrates sparse attention, model quantization, and speculative
sampling to achieve efficient prefilling and decoding. To meet diverse
on-device requirements, MiniCPM4 is available in two versions, with 0.5B and 8B
parameters, respectively. Sufficient evaluation results show that MiniCPM4
outperforms open-source models of similar size across multiple benchmarks,
highlighting both its efficiency and effectiveness. Notably, MiniCPM4-8B
demonstrates significant speed improvements over Qwen3-8B when processing long
sequences. Through further adaptation, MiniCPM4 successfully powers diverse
applications, including trustworthy survey generation and tool use with model
context protocol, clearly showcasing its broad usability.
Existing safety assurance research has primarily focused on training-phase
alignment to instill safe behaviors into LLMs. However, recent studies have
exposed these methods' susceptibility to diverse jailbreak attacks.
Concurrently, inference scaling has significantly advanced LLM reasoning
capabilities but remains unexplored in the context of safety assurance.
Addressing this gap, our work pioneers inference scaling for robust and
effective LLM safety against emerging threats. We reveal that conventional
inference scaling techniques, despite their success in reasoning tasks, perform
poorly in safety contexts, even falling short of basic approaches like
Best-of-N Sampling. We attribute this inefficiency to a newly identified
challenge, the exploration--efficiency dilemma, arising from the high
computational overhead associated with frequent process reward model (PRM)
evaluations. To overcome this dilemma, we propose SAFFRON, a novel inference
scaling paradigm tailored explicitly for safety assurance. Central to our
approach is the introduction of a multifurcation reward model (MRM) that
significantly reduces the required number of reward model evaluations. To
operationalize this paradigm, we further propose: (i) a partial supervision
training objective for MRM, (ii) a conservative exploration constraint to
prevent out-of-distribution explorations, and (iii) a Trie-based key--value
caching strategy that facilitates cache sharing across sequences during tree
search. Extensive experiments validate the effectiveness of our method.
Additionally, we publicly release our trained multifurcation reward model
(Saffron-1) and the accompanying token-level safety reward dataset (Safety4M)
to accelerate future research in LLM safety. Our code, model, and data are
publicly available at https://github.com/q-rz/saffron , and our project
homepage is at https://q-rz.github.io/p/saffron .
Text-to-image (T2I) models have garnered significant attention for generating
high-quality images aligned with text prompts. However, rapid T2I model
advancements reveal limitations in early benchmarks, lacking comprehensive
evaluations, for example, the evaluation on reasoning, text rendering and
style. Notably, recent state-of-the-art models, with their rich knowledge
modeling capabilities, show promising results on the image generation problems
requiring strong reasoning ability, yet existing evaluation systems have not
adequately addressed this frontier. To systematically address these gaps, we
introduce OneIG-Bench, a meticulously designed comprehensive benchmark
framework for fine-grained evaluation of T2I models across multiple dimensions,
including prompt-image alignment, text rendering precision, reasoning-generated
content, stylization, and diversity. By structuring the evaluation, this
benchmark enables in-depth analysis of model performance, helping researchers
and practitioners pinpoint strengths and bottlenecks in the full pipeline of
image generation. Specifically, OneIG-Bench enables flexible evaluation by
allowing users to focus on a particular evaluation subset. Instead of
generating images for the entire set of prompts, users can generate images only
for the prompts associated with the selected dimension and complete the
corresponding evaluation accordingly. Our codebase and dataset are now publicly
available to facilitate reproducible evaluation studies and cross-model
comparisons within the T2I research community.
SpatialLM is a large language model designed to process 3D point cloud data
and generate structured 3D scene understanding outputs. These outputs include
architectural elements like walls, doors, windows, and oriented object boxes
with their semantic categories. Unlike previous methods which exploit
task-specific network designs, our model adheres to the standard multimodal LLM
architecture and is fine-tuned directly from open-source LLMs.
To train SpatialLM, we collect a large-scale, high-quality synthetic dataset
consisting of the point clouds of 12,328 indoor scenes (54,778 rooms) with
ground-truth 3D annotations, and conduct a careful study on various modeling
and training decisions. On public benchmarks, our model gives state-of-the-art
performance in layout estimation and competitive results in 3D object
detection. With that, we show a feasible path for enhancing the spatial
understanding capabilities of modern LLMs for applications in augmented
reality, embodied robotics, and more.
Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov
272
Vision encoders are increasingly used in modern applications, from
vision-only models to multimodal systems such as vision-language models.
Despite their remarkable success, it remains unclear how these architectures
represent features internally. Here, we propose a novel approach for
interpreting vision features via image reconstruction. We compare two related
model families, SigLIP and SigLIP2, which differ only in their training
objective, and show that encoders pre-trained on image-based tasks retain
significantly more image information than those trained on non-image tasks such
as contrastive learning. We further apply our method to a range of vision
encoders, ranking them by the informativeness of their feature representations.
Finally, we demonstrate that manipulating the feature space yields predictable
changes in reconstructed images, revealing that orthogonal rotations (rather
than spatial transformations) control color encoding. Our approach can be
applied to any vision encoder, shedding light on the inner structure of its
feature space. The code and model weights to reproduce the experiments are
available in GitHub.
Sheng Chen, Peiyu He, Jiaxin Hu, Ziyang Liu, Yansheng Wang, Tao Xu, Chi Zhang, Chongchong Zhang, Chao An, Shiyu Cai, Duo Cao, Kangping Chen, Shuai Chu, Tianwei Chu, Mingdi Dan, Min Du, Weiwei Fang, Pengyou Fu, Junkai Hu, Xiaowei Jiang, Zhaodi Jiang, Fuxuan Li, Jun Li, Minghui Li, Mingyao Li, Yanchang Li, Zhibin Li, Guangming Liu, Kairui Liu, Lihao Liu, Weizhi Liu, Xiaoshun Liu, Yufei Liu, Yunfei Liu, Qiang Lu, Yuanfei Luo, Xiang Lv, Hongying Ma, Sai Ma, Lingxian Mi, Sha Sa, Hongxiang Shu, Lei Tian, Chengzhi Wang, Jiayu Wang, Kaijie Wang, Qingyi Wang, Renwen Wang, Tao Wang, Wei Wang, Xirui Wang, Chao Wei, Xuguang Wei, Zijun Xia, Zhaohao Xiao, Tingshuai Yan, Liyan Yang, Yifan Yang, Zhikai Yang, Zhong Yin, Li Yuan, Liuchun Yuan, Chi Zhang, Jinyang Zhang, Junhui Zhang, Linge Zhang, Zhenyi Zhang, Zheyu Zhang, Dongjie Zhu, Hang Li, Yangang Zhang
272
Modern robot navigation systems encounter difficulties in diverse and complex
indoor environments. Traditional approaches rely on multiple modules with small
models or rule-based systems and thus lack adaptability to new environments. To
address this, we developed Astra, a comprehensive dual-model architecture,
Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a
multimodal LLM, processes vision and language inputs to perform self and goal
localization using a hybrid topological-semantic graph as the global map, and
outperforms traditional visual place recognition methods. Astra-Local, a
multitask network, handles local path planning and odometry estimation. Its 4D
spatial-temporal encoder, trained through self-supervised learning, generates
robust 4D features for downstream tasks. The planning head utilizes flow
matching and a novel masked ESDF loss to minimize collision risks for
generating local trajectories, and the odometry head integrates multi-sensor
inputs via a transformer encoder to predict the relative pose of the robot.
Deployed on real in-house mobile robots, Astra achieves high end-to-end mission
success rate across diverse indoor environments.
Yijia Dai, Zhaolin Gao, Yahya Satter, Sarah Dean, Jennifer J. Sun
193
Hidden Markov Models (HMMs) are foundational tools for modeling sequential
data with latent Markovian structure, yet fitting them to real-world data
remains computationally challenging. In this work, we show that pre-trained
large language models (LLMs) can effectively model data generated by HMMs via
in-context learning (ICL)x2013their ability to infer patterns from
examples within a prompt. On a diverse set of synthetic HMMs, LLMs achieve
predictive accuracy approaching the theoretical optimum. We uncover novel
scaling trends influenced by HMM properties, and offer theoretical conjectures
for these empirical observations. We also provide practical guidelines for
scientists on using ICL as a diagnostic tool for complex data. On real-world
animal decision-making tasks, ICL achieves competitive performance with models
designed by human experts. To our knowledge, this is the first demonstration
that ICL can learn and predict HMM-generated sequencesx2013an
advance that deepens our understanding of in-context learning in LLMs and
establishes its potential as a powerful tool for uncovering hidden structure in
complex scientific data.
Vision-Language-Action (VLA) models have shown impressive capabilities across
a wide range of robotics manipulation tasks. However, their growing model size
poses significant challenges for deployment on resource-constrained robotic
systems. While 1-bit pretraining has proven effective for enhancing the
inference efficiency of large language models with minimal performance loss,
its application to VLA models remains underexplored. In this work, we present
BitVLA, the first 1-bit VLA model for robotics manipulation, in which every
parameter is ternary, i.e., {-1, 0, 1}. To further reduce the memory footprint
of the vision encoder, we propose the distillation-aware training strategy that
compresses the full-precision encoder to 1.58-bit weights. During this process,
a full-precision encoder serves as a teacher model to better align latent
representations. Despite the lack of large-scale robotics pretraining, BitVLA
achieves performance comparable to the state-of-the-art model OpenVLA-OFT with
4-bit post-training quantization on the LIBERO benchmark, while consuming only
29.8% of the memory. These results highlight BitVLA's promise for deployment on
memory-constrained edge devices. We release the code and model weights in
https://github.com/ustcwhy/BitVLA.
Long chain-of-thought (CoT) supervision has become a common strategy to
enhance reasoning in language models. While effective for large models, we
identify a phenomenon we call Long CoT Degradation, in which small language
models (SLMs; <=3B parameters) trained on limited long CoT data experience
significant performance deterioration. Through extensive experiments on the
Qwen2.5, LLaMA3 and Gemma3 families, we demonstrate that this degradation is
widespread across SLMs. In some settings, models trained on only 8k long CoT
examples lose up to 75% of their original performance before fine-tuning.
Strikingly, we further observe that for some particularly small models, even
training on 220k long CoT examples fails to recover or surpass their original
performance prior to fine-tuning. Our analysis attributes this effect to error
accumulation: while longer responses increase the capacity for multi-step
reasoning, they also amplify the risk of compounding mistakes. Furthermore, we
find that Long CoT Degradation may negatively impacts downstream reinforcement
learning (RL), although this can be alleviated by sufficiently scaled
supervised fine-tuning (SFT). Our findings challenge common assumptions about
the benefits of long CoT training for SLMs and offer practical guidance for
building more effective small-scale reasoning models.
Zhengyao Lv, Tianlin Pan, Chenyang Si, Zhaoxi Chen, Wangmeng Zuo, Ziwei Liu, Kwan-Yee K. Wong
152
Multimodal Diffusion Transformers (MM-DiTs) have achieved remarkable progress
in text-driven visual generation. However, even state-of-the-art MM-DiT models
like FLUX struggle with achieving precise alignment between text prompts and
generated content. We identify two key issues in the attention mechanism of
MM-DiT, namely 1) the suppression of cross-modal attention due to token
imbalance between visual and textual modalities and 2) the lack of
timestep-aware attention weighting, which hinder the alignment. To address
these issues, we propose Temperature-Adjusted Cross-modal Attention
(TACA), a parameter-efficient method that dynamically rebalances multimodal
interactions through temperature scaling and timestep-dependent adjustment.
When combined with LoRA fine-tuning, TACA significantly enhances text-image
alignment on the T2I-CompBench benchmark with minimal computational overhead.
We tested TACA on state-of-the-art models like FLUX and SD3.5, demonstrating
its ability to improve image-text alignment in terms of object appearance,
attribute binding, and spatial relationships. Our findings highlight the
importance of balancing cross-modal attention in improving semantic fidelity in
text-to-image diffusion models. Our codes are publicly available at
https://github.com/Vchitect/TACA
Nick Jiang, Amil Dravid, Alexei Efros, Yossi Gandelsman
142
We investigate the mechanism underlying a previously identified phenomenon in
Vision Transformers -- the emergence of high-norm tokens that lead to noisy
attention maps. We observe that in multiple models (e.g., CLIP, DINOv2), a
sparse set of neurons is responsible for concentrating high-norm activations on
outlier tokens, leading to irregular attention patterns and degrading
downstream visual processing. While the existing solution for removing these
outliers involves retraining models from scratch with additional learned
register tokens, we use our findings to create a training-free approach to
mitigate these artifacts. By shifting the high-norm activations from our
discovered register neurons into an additional untrained token, we can mimic
the effect of register tokens on a model already trained without registers. We
demonstrate that our method produces cleaner attention and feature maps,
enhances performance over base models across multiple downstream visual tasks,
and achieves results comparable to models explicitly trained with register
tokens. We then extend test-time registers to off-the-shelf vision-language
models to improve their interpretability. Our results suggest that test-time
registers effectively take on the role of register tokens at test-time,
offering a training-free solution for any pre-trained model released without
them.
Developing generalizable reasoning capabilities in multimodal large language
models (MLLMs) remains challenging. Motivated by cognitive science literature
suggesting that gameplay promotes transferable cognitive skills, we propose a
novel post-training paradigm, Visual Game Learning, or ViGaL, where MLLMs
develop out-of-domain generalization of multimodal reasoning through playing
arcade-like games. Specifically, we show that post-training a 7B-parameter MLLM
via reinforcement learning (RL) on simple arcade-like games, e.g. Snake,
significantly enhances its downstream performance on multimodal math benchmarks
like MathVista, and on multi-discipline questions like MMMU, without seeing any
worked solutions, equations, or diagrams during RL, suggesting the capture of
transferable reasoning skills. Remarkably, our model outperforms specialist
models tuned on multimodal reasoning data in multimodal reasoning benchmarks,
while preserving the base model's performance on general visual benchmarks, a
challenge where specialist models often fall short. Our findings suggest a new
post-training paradigm: synthetic, rule-based games can serve as controllable
and scalable pre-text tasks that unlock generalizable multimodal reasoning
abilities in MLLMs.
Noy Sternlicht, Ariel Gera, Roy Bar-Haim, Tom Hope, Noam Slonim
132
We introduce Debate Speech Evaluation as a novel and challenging benchmark
for assessing LLM judges. Evaluating debate speeches requires a deep
understanding of the speech at multiple levels, including argument strength and
relevance, the coherence and organization of the speech, the appropriateness of
its style and tone, and so on. This task involves a unique set of cognitive
abilities that have previously received limited attention in systematic LLM
benchmarking. To explore such skills, we leverage a dataset of over 600
meticulously annotated debate speeches and present the first in-depth analysis
of how state-of-the-art LLMs compare to human judges on this task. Our findings
reveal a nuanced picture: while larger models can approximate individual human
judgments in some respects, they differ substantially in their overall judgment
behavior. We also investigate the ability of frontier LLMs to generate
persuasive, opinionated speeches, showing that models may perform at a human
level on this task.
Optical Chemical Structure Recognition (OCSR) is crucial for digitizing
chemical knowledge by converting molecular images into machine-readable
formats. While recent vision-language models (VLMs) have shown potential in
this task, their image-captioning approach often struggles with complex
molecular structures and inconsistent annotations. To overcome these
challenges, we introduce GTR-Mol-VLM, a novel framework featuring two key
innovations: (1) the Graph Traversal as Visual Chain of Thought
mechanism that emulates human reasoning by incrementally parsing molecular
graphs through sequential atom-bond predictions, and (2) the data-centric
principle of Faithfully Recognize What You've Seen, which addresses
the mismatch between abbreviated structures in images and their expanded
annotations. To support model development, we constructed GTR-CoT-1.3M, a
large-scale instruction-tuning dataset with meticulously corrected annotations,
and introduced MolRec-Bench, the first benchmark designed for a fine-grained
evaluation of graph-parsing accuracy in OCSR. Comprehensive experiments
demonstrate that GTR-Mol-VLM achieves superior results compared to specialist
models, chemistry-domain VLMs, and commercial general-purpose VLMs. Notably, in
scenarios involving molecular images with functional group abbreviations,
GTR-Mol-VLM outperforms the second-best baseline by approximately 14 percentage
points, both in SMILES-based and graph-based metrics. We hope that this work
will drive OCSR technology to more effectively meet real-world needs, thereby
advancing the fields of cheminformatics and AI for Science. We will release
GTR-CoT at https://github.com/opendatalab/GTR-CoT.
Recent generations of language models have introduced Large Reasoning Models
(LRMs) that generate detailed thinking processes before providing answers.
While these models demonstrate improved performance on reasoning benchmarks,
their fundamental capabilities, scaling properties, and limitations remain
insufficiently understood. Current evaluations primarily focus on established
math and coding benchmarks, emphasizing final answer accuracy. However, this
evaluation paradigm often suffers from contamination and does not provide
insights into the reasoning traces. In this work, we systematically investigate
these gaps with the help of controllable puzzle environments that allow precise
manipulation of complexity while maintaining consistent logical structures.
This setup enables the analysis of not only final answers but also the internal
reasoning traces, offering insights into how LRMs think. Through extensive
experiments, we show that LRMs face a complete accuracy collapse beyond certain
complexities. Moreover, they exhibit a counterintuitive scaling limit: their
reasoning effort increases with problem complexity up to a point, then declines
despite having remaining token budget. By comparing LRMs with their standard
LLM counterparts under same inference compute, we identify three performance
regimes: (1) low-complexity tasks where standard models outperform LRMs, (2)
medium-complexity tasks where LRMs demonstrates advantage, and (3)
high-complexity tasks where both models face complete collapse. We found that
LRMs have limitations in exact computation: they fail to use explicit
algorithms and reason inconsistently across scales. We also investigate the
reasoning traces in more depth, studying the patterns of explored solutions and
analyzing the models' computational behavior, shedding light on their
strengths, limitations, and raising questions about their reasoning
capabilities.
Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B. Cohen, Edoardo M. Ponti
112
To what extent do vision-and-language foundation models possess a realistic
world model (observation times action rightarrow observation) and a
dynamics model (observation times observation rightarrow action), when
actions are expressed through language? While open-source foundation models
struggle with both, we find that fine-tuning them to acquire a dynamics model
through supervision is significantly easier than acquiring a world model. In
turn, dynamics models can be used to bootstrap world models through two main
strategies: 1) weakly supervised learning from synthetic data and 2) inference
time verification. Firstly, the dynamics model can annotate actions for
unlabelled pairs of video frame observations to expand the training data. We
further propose a new objective, where image tokens in observation pairs are
weighted by their importance, as predicted by a recognition model. Secondly,
the dynamics models can assign rewards to multiple samples of the world model
to score them, effectively guiding search at inference time. We evaluate the
world models resulting from both strategies through the task of action-centric
image editing on Aurora-Bench. Our best model achieves a performance
competitive with state-of-the-art image editing models, improving on them by a
margin of 15% on real-world subsets according to GPT4o-as-judge, and
achieving the best average human evaluation across all subsets of Aurora-Bench.
Yin Huang, Yifan Ethan Xu, Kai Sun, Vera Yan, Alicia Sun, Haidar Khan, Jimmy Nguyen, Mohammad Kachuee, Zhaojiang Lin, Yue Liu, Aaron Colak, Anuj Kumar, Wen-tau Yih, Xin Luna Dong
92
Can we teach Large Language Models (LLMs) to refrain from hallucinating
factual statements? In this paper we present a fine-tuning strategy that we
call ConfQA, which can reduce hallucination rate from 20-40% to under 5% across
multiple factuality benchmarks. The core idea is simple: when the LLM answers a
question correctly, it is trained to continue with the answer; otherwise, it is
trained to admit "I am unsure". But there are two key factors that make the
training highly effective. First, we introduce a dampening prompt "answer only
if you are confident" to explicitly guide the behavior, without which
hallucination remains high as 15%-25%. Second, we leverage simple factual
statements, specifically attribute values from knowledge graphs, to help LLMs
calibrate the confidence, resulting in robust generalization across domains and
question types. Building on this insight, we propose the Dual Neural Knowledge
framework, which seamlessly select between internally parameterized neural
knowledge and externally recorded symbolic knowledge based on ConfQA's
confidence. The framework enables potential accuracy gains to beyond 95%, while
reducing unnecessary external retrievals by over 30%.
Guang Liu, Liangdong Wang, Jijie Li, Yang Yu, Yao Xu, Jiabei Chen, Yu Bai, Feng Liao, Yonghua Lin
82
We introduce CCI4.0, a large-scale bilingual pre-training dataset engineered
for superior data quality and diverse human-like reasoning trajectory. CCI4.0
occupies roughly 35 TB of disk space and comprises two sub-datasets:
CCI4.0-M2-Base and CCI4.0-M2-CoT. CCI4.0-M2-Base combines a 5.2 TB carefully
curated Chinese web corpus, a 22.5 TB English subset from Nemotron-CC, and
diverse sources from math, wiki, arxiv, and code. Although these data are
mostly sourced from well-processed datasets, the quality standards of various
domains are dynamic and require extensive expert experience and labor to
process. So, we propose a novel pipeline justifying data quality mainly based
on models through two-stage deduplication, multiclassifier quality scoring, and
domain-aware fluency filtering. We extract 4.5 billion pieces of
CoT(Chain-of-Thought) templates, named CCI4.0-M2-CoT. Differing from the
distillation of CoT from larger models, our proposed staged CoT extraction
exemplifies diverse reasoning patterns and significantly decreases the
possibility of hallucination. Empirical evaluations demonstrate that LLMs
pre-trained in CCI4.0 benefit from cleaner, more reliable training signals,
yielding consistent improvements in downstream tasks, especially in math and
code reflection tasks. Our results underscore the critical role of rigorous
data curation and human thinking templates in advancing LLM performance,
shedding some light on automatically processing pretraining corpora.
Jie Ruan, Inderjeet Nair, Shuyang Cao, Amy Liu, Sheza Munir, Micah Pollens-Dempsey, Tiffany Chiang, Lucy Kates, Nicholas David, Sihan Chen, Ruxin Yang, Yuqian Yang, Jasmine Gump, Tessa Bialek, Vivek Sankaran, Margo Schlanger, Lu Wang
82
This paper introduces ExpertLongBench, an expert-level benchmark containing
11 tasks from 9 domains that reflect realistic expert workflows and
applications. Beyond question answering, the application-driven tasks in
ExpertLongBench demand long-form outputs that can exceed 5,000 tokens and
strict adherence to domain-specific requirements. Notably, each task in
ExpertLongBench includes a rubric, designed or validated by domain experts, to
specify task requirements and guide output evaluation. Furthermore, we propose
CLEAR, an evaluation framework that supports accurate evaluation of long-form
model outputs in our benchmark. To achieve fine-grained, expert-aligned
evaluation, CLEAR derives checklists from both model outputs and references by
extracting information corresponding to items in the task-specific rubric.
Checklist items for model outputs are then compared with corresponding items
for reference outputs to assess their correctness, enabling grounded
evaluation. We benchmark 11 large language models (LLMs) and analyze components
in CLEAR, showing that (1) existing LLMs, with the top performer achieving only
a 26.8% F1 score, require significant improvement for expert-level tasks; (2)
models can generate content corresponding to the required aspects, though often
not accurately; and (3) accurate checklist extraction and comparison in CLEAR
can be achieved by open-weight models for more scalable and low-cost usage.
Amber Yijia Zheng, Cedar Site Bai, Brian Bullins, Raymond A. Yeh
82
Model immunization aims to pre-train models that are difficult to fine-tune
on harmful tasks while retaining their utility on other non-harmful tasks.
Though prior work has shown empirical evidence for immunizing text-to-image
models, the key understanding of when immunization is possible and a precise
definition of an immunized model remain unclear. In this work, we propose a
framework, based on the condition number of a Hessian matrix, to analyze model
immunization for linear models. Building on this framework, we design an
algorithm with regularization terms to control the resulting condition numbers
after pre-training. Empirical results on linear models and non-linear deep-nets
demonstrate the effectiveness of the proposed algorithm on model immunization.
The code is available at
https://github.com/amberyzheng/model-immunization-cond-num.
Penghao Wu, Shengnan Ma, Bo Wang, Jiaheng Yu, Lewei Lu, Ziwei Liu
72
Multimodal Large Language Models (MLLMs) have shown great potential in
revolutionizing Graphical User Interface (GUI) automation. However, existing
GUI models mostly rely on learning from nearly error-free offline trajectories,
thus lacking reflection and error recovery capabilities. To bridge this gap, we
propose GUI-Reflection, a novel framework that explicitly integrates
self-reflection and error correction capabilities into end-to-end multimodal
GUI models throughout dedicated training stages: GUI-specific pre-training,
offline supervised fine-tuning (SFT), and online reflection tuning.
GUI-reflection enables self-reflection behavior emergence with fully automated
data generation and learning processes without requiring any human annotation.
Specifically, 1) we first propose scalable data pipelines to automatically
construct reflection and error correction data from existing successful
trajectories. While existing GUI models mainly focus on grounding and UI
understanding ability, we propose the GUI-Reflection Task Suite to learn and
evaluate reflection-oriented abilities explicitly. 2) Furthermore, we built a
diverse and efficient environment for online training and data collection of
GUI models on mobile devices. 3) We also present an iterative online reflection
tuning algorithm leveraging the proposed environment, enabling the model to
continuously enhance its reflection and error correction abilities. Our
framework equips GUI agents with self-reflection and correction capabilities,
paving the way for more robust, adaptable, and intelligent GUI automation, with
all data, models, environments, and tools to be released publicly.
Large-scale video generative models can synthesize diverse and realistic
visual content for dynamic world creation, but they often lack element-wise
controllability, hindering their use in editing scenes and training embodied AI
agents. We propose Dreamland, a hybrid world generation framework combining the
granular control of a physics-based simulator and the photorealistic content
output of large-scale pretrained generative models. In particular, we design a
layered world abstraction that encodes both pixel-level and object-level
semantics and geometry as an intermediate representation to bridge the
simulator and the generative model. This approach enhances controllability,
minimizes adaptation cost through early alignment with real-world
distributions, and supports off-the-shelf use of existing and future pretrained
generative models. We further construct a D3Sim dataset to facilitate the
training and evaluation of hybrid generation pipelines. Experiments demonstrate
that Dreamland outperforms existing baselines with 50.8% improved image
quality, 17.9% stronger controllability, and has great potential to enhance
embodied agent training. Code and data will be made available.
Large Language Models (LLMs) require alignment with human preferences to
avoid generating offensive, false, or meaningless content. Recently,
low-resource methods for LLM alignment have been popular, while still facing
challenges in obtaining both high-quality and aligned content. Motivated by the
observation that the difficulty of generating aligned responses is concentrated
at the beginning of decoding, we propose a novel framework, Weak-to-Strong
Decoding (WSD), to enhance the alignment ability of base models by the guidance
of a small aligned model. The small model first drafts well-aligned beginnings,
followed by the large base model to continue the rest, controlled by a
well-designed auto-switch mechanism. We also collect a new dataset, GenerAlign,
to fine-tune a small-sized Pilot-3B as the draft model, which effectively
enhances different base models under the WSD framework to outperform all
baseline methods, while avoiding degradation on downstream tasks, termed as the
alignment tax. Extensive experiments are further conducted to examine the
impact of different settings and time efficiency, as well as analyses on the
intrinsic mechanisms of WSD in depth.
Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Held, Diyi Yang
62
Recent calls for pluralistic alignment of Large Language Models (LLMs)
encourage adapting models to diverse user preferences. However, most prior work
on personalized reward models heavily rely on additional identity information,
such as demographic details or a predefined set of preference categories. To
this end, we introduce SynthesizeMe, an approach to inducing synthetic user
personas from user interactions for personalized reward modeling. SynthesizeMe
first generates and verifies reasoning to explain user preferences, then
induces synthetic user personas from that reasoning, and finally filters to
informative prior user interactions in order to build personalized prompts for
a particular user. We show that using SynthesizeMe induced prompts improves
personalized LLM-as-a-judge accuracy by 4.4% on Chatbot Arena. Combining
SynthesizeMe derived prompts with a reward model achieves top performance on
PersonalRewardBench: a new curation of user-stratified interactions with
chatbots collected from 854 users of Chatbot Arena and PRISM.
Recent advances in large language models (LLMs) and vision-language models
(VLMs) have enabled powerful autonomous agents capable of complex reasoning and
multi-modal tool use. Despite their growing capabilities, today's agent
frameworks remain fragile, lacking principled mechanisms for secure information
flow, reliability, and multi-agent coordination. In this work, we introduce
SAFEFLOW, a new protocol-level framework for building trustworthy LLM/VLM-based
agents. SAFEFLOW enforces fine-grained information flow control (IFC),
precisely tracking provenance, integrity, and confidentiality of all the data
exchanged between agents, tools, users, and environments. By constraining LLM
reasoning to respect these security labels, SAFEFLOW prevents untrusted or
adversarial inputs from contaminating high-integrity decisions. To ensure
robustness in concurrent multi-agent settings, SAFEFLOW introduces
transactional execution, conflict resolution, and secure scheduling over shared
state, preserving global consistency across agents. We further introduce
mechanisms, including write-ahead logging, rollback, and secure caches, that
further enhance resilience against runtime errors and policy violations. To
validate the performances, we built SAFEFLOWBENCH, a comprehensive benchmark
suite designed to evaluate agent reliability under adversarial, noisy, and
concurrent operational conditions. Extensive experiments demonstrate that
agents built with SAFEFLOW maintain impressive task performance and security
guarantees even in hostile environments, substantially outperforming
state-of-the-art. Together, SAFEFLOW and SAFEFLOWBENCH lay the groundwork for
principled, robust, and secure agent ecosystems, advancing the frontier of
reliable autonomy.
Large language models frequently rely on both contextual input and parametric
knowledge to perform tasks. However, these sources can come into conflict,
especially when retrieved documents contradict the model's parametric
knowledge. We propose a diagnostic framework to systematically evaluate LLM
behavior under context-memory conflict, where the contextual information
diverges from their parametric beliefs. We construct diagnostic data that
elicit these conflicts and analyze model performance across multiple task
types. Our findings reveal that (1) knowledge conflict has minimal impact on
tasks that do not require knowledge utilization, (2) model performance is
consistently higher when contextual and parametric knowledge are aligned, (3)
models are unable to fully suppress their internal knowledge even when
instructed, and (4) providing rationales that explain the conflict increases
reliance on contexts. These insights raise concerns about the validity of
model-based evaluation and underscore the need to account for knowledge
conflict in the deployment of LLMs.
Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, Will Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, Christopher Re
52
Large language models are often used to answer queries grounded in large text
corpora (e.g. codebases, legal documents, or chat histories) by placing the
entire corpus in the context window and leveraging in-context learning (ICL).
Although current models support contexts of 100K-1M tokens, this setup is
costly to serve because the memory consumption of the KV cache scales with
input length. We explore an alternative: training a smaller KV cache offline on
each corpus. At inference time, we load this trained KV cache, which we call a
Cartridge, and decode a response. Critically, the cost of training a Cartridge
can be amortized across all the queries referencing the same corpus. However,
we find that the naive approach of training the Cartridge with next-token
prediction on the corpus is not competitive with ICL. Instead, we propose
self-study, a training recipe in which we generate synthetic conversations
about the corpus and train the Cartridge with a context-distillation objective.
We find that Cartridges trained with self-study replicate the functionality of
ICL, while being significantly cheaper to serve. On challenging long-context
benchmarks, Cartridges trained with self-study match ICL performance while
using 38.6x less memory and enabling 26.4x higher throughput. Self-study also
extends the model's effective context length (e.g. from 128k to 484k tokens on
MTOB) and surprisingly, leads to Cartridges that can be composed at inference
time without retraining.
Nikolas Belle, Dakota Barnes, Alfonso Amayuelas, Ivan Bercovich, Xin Eric Wang, William Wang
52
Recent advances in LLMs have enabled their use as autonomous agents across a
range of tasks, yet they continue to struggle with formulating and adhering to
coherent long-term strategies. In this paper, we investigate whether LLM agents
can self-improve when placed in environments that explicitly challenge their
strategic planning abilities. Using the board game Settlers of Catan, accessed
through the open-source Catanatron framework, we benchmark a progression of
LLM-based agents, from a simple game-playing agent to systems capable of
autonomously rewriting their own prompts and their player agent's code. We
introduce a multi-agent architecture in which specialized roles (Analyzer,
Researcher, Coder, and Player) collaborate to iteratively analyze gameplay,
research new strategies, and modify the agent's logic or prompt. By comparing
manually crafted agents to those evolved entirely by LLMs, we evaluate how
effectively these systems can diagnose failure and adapt over time. Our results
show that self-evolving agents, particularly when powered by models like Claude
3.7 and GPT-4o, outperform static baselines by autonomously adopting their
strategies, passing along sample behavior to game-playing agents, and
demonstrating adaptive reasoning over multiple iterations.
In this work, we address dynamic view synthesis from monocular videos as an
inverse problem in a training-free setting. By redesigning the noise
initialization phase of a pre-trained video diffusion model, we enable
high-fidelity dynamic view synthesis without any weight updates or auxiliary
modules. We begin by identifying a fundamental obstacle to deterministic
inversion arising from zero-terminal signal-to-noise ratio (SNR) schedules and
resolve it by introducing a novel noise representation, termed K-order
Recursive Noise Representation. We derive a closed form expression for this
representation, enabling precise and efficient alignment between the
VAE-encoded and the DDIM inverted latents. To synthesize newly visible regions
resulting from camera motion, we introduce Stochastic Latent Modulation, which
performs visibility aware sampling over the latent space to complete occluded
regions. Comprehensive experiments demonstrate that dynamic view synthesis can
be effectively performed through structured latent manipulation in the noise
initialization phase.
Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik Narasimhan
42
Existing benchmarks for conversational AI agents simulate single-control
environments, where only the AI agent can use tools to interact with the world,
while the user remains a passive information provider. This differs from
real-world scenarios like technical support, where users need to actively
participate in modifying the state of the (shared) world. In order to address
this gap, we introduce tau^2-bench, with four key contributions:
1) A novel Telecom dual-control domain modeled as a Dec-POMDP, where both
agent and user make use of tools to act in a shared, dynamic environment that
tests both agent coordination and communication,
2) A compositional task generator that programmatically creates diverse,
verifiable tasks from atomic components, ensuring domain coverage and
controlled complexity,
3) A reliable user simulator tightly coupled with the environment, whose
behavior is constrained by tools and observable states, improving simulation
fidelity,
4) Fine-grained analysis of agent performance through multiple ablations
including separating errors arising from reasoning vs
communication/coordination.
In particular, our experiments show significant performance drops when agents
shift from no-user to dual-control, highlighting the challenges of guiding
users. Overall, tau^2-bench provides a controlled testbed for agents that
must both reason effectively and guide user actions.
Current Multimodal Large Language Models (MLLMs) may struggle with
understanding long or complex videos due to computational demands at test time,
lack of robustness, and limited accuracy, primarily stemming from their
feed-forward processing nature. These limitations could be more severe for
models with fewer parameters. To address these limitations, we propose a novel
framework inspired by cybernetic principles, redesigning video MLLMs as
adaptive systems capable of self-monitoring, self-correction, and dynamic
resource allocation during inference. Our approach, CyberV, introduces a
cybernetic loop consisting of an MLLM Inference System, a Sensor, and a
Controller. Specifically, the sensor monitors forward processes of the MLLM and
collects intermediate interpretations, such as attention drift, then the
controller determines when and how to trigger self-correction and generate
feedback to guide the next round. This test-time adaptive scaling framework
enhances frozen MLLMs without requiring retraining or additional components.
Experiments demonstrate significant improvements: CyberV boosts Qwen2.5-VL-7B
by 8.3% and InternVL3-8B by 5.5% on VideoMMMU, surpassing the competitive
proprietary model GPT-4o. When applied to Qwen2.5-VL-72B, it yields a 10.0%
improvement, achieving performance even comparable to human experts.
Furthermore, our method demonstrates consistent gains on general-purpose
benchmarks, such as VideoMME and WorldSense, highlighting its effectiveness and
generalization capabilities in making MLLMs more robust and accurate for
dynamic video understanding. The code is released at
https://github.com/marinero4972/CyberV.
Michael K. Chen, Xikun Zhang, Jiaxing Huang, Dacheng Tao
32
Large language models (LLMs) have become the cornerstone of modern AI.
However, the existing paradigm of next-token prediction fundamentally limits
their ability to form coherent, high-level concepts, making it a critical
barrier to human-like understanding and reasoning. Take the phrase "ribonucleic
acid" as an example: an LLM will first decompose it into tokens, i.e.,
artificial text fragments ("rib", "on", ...), then learn each token
sequentially, rather than grasping the phrase as a unified, coherent semantic
entity. This fragmented representation hinders deeper conceptual understanding
and, ultimately, the development of truly intelligent systems. In response, we
introduce Concept-Aware Fine-Tuning (CAFT), a novel multi-token training method
that redefines how LLMs are fine-tuned. By enabling the learning of sequences
that span multiple tokens, this method fosters stronger concept-aware learning.
Our experiments demonstrate significant improvements compared to conventional
next-token finetuning methods across diverse tasks, including traditional
applications like text summarization and domain-specific ones like de novo
protein design. Multi-token prediction was previously only possible in the
prohibitively expensive pretraining phase; CAFT, to our knowledge, is the first
to bring the multi-token setting to the post-training phase, thus effectively
democratizing its benefits for the broader community of practitioners and
researchers. Finally, the unexpected effectiveness of our proposed method
suggests wider implications for the machine learning research community. All
code and data are available at https://github.com/michaelchen-lab/caft-llm
Recent advances in large language model (LLM) reasoning have shown that
sophisticated behaviors such as planning and self-reflection can emerge through
reinforcement learning (RL). However, despite these successes, RL in its
current form remains insufficient to induce capabilities that exceed the
limitations of the base model, as it is primarily optimized based on existing
knowledge of the model rather than facilitating the acquisition of new
information. To address this limitation, we employ supervised fine-tuning (SFT)
to learn what RL cannot, which enables the incorporation of new knowledge and
reasoning patterns by leveraging high-quality demonstration data. We analyze
the training dynamics of RL and SFT for LLM reasoning and find that RL excels
at maintaining and improving performance on questions within the model's
original capabilities, while SFT is more effective at enabling progress on
questions beyond the current scope of the model. Motivated by the complementary
strengths of RL and SFT, we introduce a novel training approach,
ReLIFT (Reinforcement Learning Interleaved
with Online Fine-Tuning). In ReLIFT, the model is primarily
trained using RL, but when it encounters challenging questions, high-quality
solutions are collected for fine-tuning, and the training process alternates
between RL and fine-tuning to enhance the model's reasoning abilities. ReLIFT
achieves an average improvement of over +5.2 points across five
competition-level benchmarks and one out-of-distribution benchmark compared to
other zero-RL models. Furthermore, we demonstrate that ReLIFT outperforms both
RL and SFT while using only 13\% of the detailed demonstration data,
highlighting its scalability. These results provide compelling evidence that
ReLIFT overcomes the fundamental limitations of RL and underscores the
significant potential.
Recently, techniques such as explicit structured reasoning have demonstrated
strong test-time scaling behavior by enforcing a separation between the model's
internal "thinking" process and the final response. A key factor influencing
answer quality in this setting is the length of the thinking stage. When the
reasoning is too short, the model may fail to capture the complexity of the
task. Conversely, when it is too long, the model may overthink, leading to
unnecessary computation and degraded performance. This paper explores and
exploits the underlying mechanisms by which LLMs understand and regulate the
length of their reasoning during explicit thought processes. First, we show
that LLMs encode their progress through the reasoning process and introduce an
interactive progress bar visualization, which is then used to reveal insights
on the model's planning dynamics. Second, we manipulate the internal progress
encoding during inference to reduce unnecessary steps and generate a more
concise and decisive chain of thoughts. Our empirical results demonstrate that
this "overclocking" method mitigates overthinking, improves answer accuracy,
and reduces inference latency. Our code is publicly available.
Recent advances in large language models (LLMs) have demonstrated remarkable
capabilities across diverse domains, particularly in mathematical reasoning,
amid which geometry problem solving remains a challenging area where auxiliary
construction plays a enssential role. Existing approaches either achieve
suboptimal performance or rely on massive LLMs (e.g., GPT-4o), incurring
massive computational costs. We posit that reinforcement learning with
verifiable reward (e.g., GRPO) offers a promising direction for training
smaller models that effectively combine auxiliary construction with robust
geometric reasoning. However, directly applying GRPO to geometric reasoning
presents fundamental limitations due to its dependence on unconditional
rewards, which leads to indiscriminate and counterproductive auxiliary
constructions. To address these challenges, we propose Group Contrastive Policy
Optimization (GCPO), a novel reinforcement learning framework featuring two key
innovations: (1) Group Contrastive Masking, which adaptively provides positive
or negative reward signals for auxiliary construction based on contextual
utility, and a (2) length reward that promotes longer reasoning chains.
Building on GCPO, we develop GeometryZero, a family of affordable-size
geometric reasoning models that judiciously determine when to employ auxiliary
construction. Our extensive empirical evaluation across popular geometric
benchmarks (Geometry3K, MathVista) demonstrates that GeometryZero models
consistently outperform baselines (e.g. GRPO), achieving an average improvement
of 4.29% across all benchmarks.
Calvin Luo, Zilai Zeng, Mingxi Jia, Yilun Du, Chen Sun
32
Video generative models trained on expert demonstrations have been utilized
as performant text-conditioned visual planners for solving robotic tasks.
However, generalization to unseen tasks remains a challenge. Whereas improved
generalization may be facilitated by leveraging learned prior knowledge from
additional pre-collected offline data sources, such as web-scale video
datasets, in the era of experience we aim to design agents that can
continuously improve in an online manner from self-collected behaviors. In this
work we thus propose the Self-Adapting Improvement Loop (SAIL), where an
in-domain video model iteratively updates itself on self-produced trajectories,
collected through adaptation with an internet-scale pretrained video model, and
steadily improves its performance for a specified task of interest. We apply
SAIL to a diverse suite of MetaWorld tasks, as well as two manipulation tasks
on a real robot arm, and find that performance improvements continuously emerge
over multiple iterations for novel tasks initially unseen during original
in-domain video model training. Furthermore, we discover that SAIL is
surprisingly robust regarding if and how the self-collected experience is
filtered, and the quality of the initial in-domain demonstrations. Through
adaptation with summarized internet-scale data, and learning through online
experience, we thus demonstrate a way to iteratively bootstrap a
high-performance video model for solving novel robotic tasks through
self-improvement.
Yajie Zhou, Jiajun Ruan, Eric S. Wang, Sadjad Fouladi, Francis Y. Yan, Kevin Hsieh, Zaoxing Liu
33
Despite growing interest in domain-specific benchmarking of large language
models (LLMs) and agents, current evaluations remain limited to static,
small-scale datasets, especially in high-stakes tasks like network operations
that demand reliability for deployments. We present NetPress, an automated
benchmark generation framework for evaluating LLM agents in network
applications. NetPress introduces a unified abstraction with state and action,
enabling dynamic generation of diverse query sets along with corresponding
ground truths. At runtime, users can specify benchmark configurations to
generate millions of queries on the fly. In addition to dynamic benchmark
construction, NetPress integrates with network emulators to provide realistic
environment feedback, supporting comprehensive evaluation across correctness,
safety, and latency. We instantiate NetPress on three representative
applications, revealing interesting fine-grained differences in agent behavior
that static, correctness-only benchmarks often miss. NetPress moves LLM
evaluation toward realistic, scalable testing in infrastructure-centric
domains, helping close the gap between benchmark performance and real-world
deployment readiness. Code is available at
https://github.com/Froot-NetSys/NetPress.
Teng Hu, Zhentao Yu, Zhengguang Zhou, Jiangning Zhang, Yuan Zhou, Qinglin Lu, Ran Yi
22
Despite recent advances in video generation, existing models still lack
fine-grained controllability, especially for multi-subject customization with
consistent identity and interaction. In this paper, we propose PolyVivid, a
multi-subject video customization framework that enables flexible and
identity-consistent generation. To establish accurate correspondences between
subject images and textual entities, we design a VLLM-based text-image fusion
module that embeds visual identities into the textual space for precise
grounding. To further enhance identity preservation and subject interaction, we
propose a 3D-RoPE-based enhancement module that enables structured
bidirectional fusion between text and image embeddings. Moreover, we develop an
attention-inherited identity injection module to effectively inject fused
identity features into the video generation process, mitigating identity drift.
Finally, we construct an MLLM-based data pipeline that combines MLLM-based
grounding, segmentation, and a clique-based subject consolidation strategy to
produce high-quality multi-subject data, effectively enhancing subject
distinction and reducing ambiguity in downstream video generation. Extensive
experiments demonstrate that PolyVivid achieves superior performance in
identity fidelity, video realism, and subject alignment, outperforming existing
open-source and commercial baselines.
We present a training-free method to transplant tokenizers in pretrained
large language models (LLMs) by reconstructing unseen token embeddings via
Orthogonal Matching Pursuit (OMP). Specifically, we approximate each
out-of-vocabulary token as a sparse linear combination of shared tokens, in two
phases: first, compute each new token's representation in the donor embedding
space with a small dictionary of shared anchor tokens, then transfer these same
sparse coefficients back into the base model's embedding space.
On two challenging cross-tokenizer tasks--LlamatoMistral NeMo (12B) and
QwentoLlama (1B)--we show that OMP achieves best zero-shot preservation of
the base model's performance across multiple benchmarks, while other zero-shot
approaches degrade significantly. Compared to baselines (zero-init, mean-init,
and existing approaches like WECHSEL, FOCUS, ZETT), OMP consistently achieves
the best overall performance, effectively bridging large tokenizer
discrepancies without gradient updates. Our analysis further identifies
mismatched numerical tokenization schemes as a critical challenge for
preserving mathematical reasoning capabilities. This technique enables direct
reuse of pretrained model weights with new tokenizers, facilitating
cross-tokenizer knowledge distillation, speculative decoding, ensembling,
merging, and domain-specific vocabulary adaptations. We integrate our method
into the open-source mergekit-tokensurgeon tool for post hoc vocabulary
realignment.
Yichi Zhang, Xin Luna Dong, Zhaojiang Lin, Andrea Madotto, Anuj Kumar, Babak Damavandi, Joyce Chai, Seungwhan Moon
22
Recent advances in conversational AI have been substantial, but developing
real-time systems for perceptual task guidance remains challenging. These
systems must provide interactive, proactive assistance based on streaming
visual inputs, yet their development is constrained by the costly and
labor-intensive process of data collection and system evaluation. To address
these limitations, we present a comprehensive framework with three key
contributions. First, we introduce a novel data curation pipeline that
synthesizes dialogues from annotated egocentric videos, resulting in \dataset,
a large-scale synthetic dialogue dataset spanning multiple domains. Second, we
develop a suite of automatic evaluation metrics, validated through extensive
human studies. Third, we propose an end-to-end model that processes streaming
video inputs to generate contextually appropriate responses, incorporating
novel techniques for handling data imbalance and long-duration videos. This
work lays the foundation for developing real-time, proactive AI assistants
capable of guiding users through diverse tasks. Project page:
https://pro-assist.github.io/
Foundational to the Chinese language and culture, Chinese characters
encompass extraordinarily extensive and ever-expanding categories, with the
latest Chinese GB18030-2022 standard containing 87,887 categories. The accurate
recognition of this vast number of characters, termed mega-category
recognition, presents a formidable yet crucial challenge for cultural heritage
preservation and digital applications. Despite significant advances in Optical
Character Recognition (OCR), mega-category recognition remains unexplored due
to the absence of comprehensive datasets, with the largest existing dataset
containing merely 16,151 categories. To bridge this critical gap, we introduce
MegaHan97K, a mega-category, large-scale dataset covering an unprecedented
97,455 categories of Chinese characters. Our work offers three major
contributions: (1) MegaHan97K is the first dataset to fully support the latest
GB18030-2022 standard, providing at least six times more categories than
existing datasets; (2) It effectively addresses the long-tail distribution
problem by providing balanced samples across all categories through its three
distinct subsets: handwritten, historical and synthetic subsets; (3)
Comprehensive benchmarking experiments reveal new challenges in mega-category
scenarios, including increased storage demands, morphologically similar
character recognition, and zero-shot learning difficulties, while also
unlocking substantial opportunities for future research. To the best of our
knowledge, the MetaHan97K is likely the dataset with the largest classes not
only in the field of OCR but may also in the broader domain of pattern
recognition. The dataset is available at
https://github.com/SCUT-DLVCLab/MegaHan97K.
Jie Sun, Junkang Wu, Jiancan Wu, Zhibo Zhu, Xingyu Lu, Jun Zhou, Lintao Ma, Xiang Wang
22
The alignment of Large Language Models (LLMs) is crucial for ensuring their
safety and reliability in practical applications. Direct Preference
Optimization (DPO) has emerged as an efficient method that directly optimizes
models using preference pairs, significantly reducing resource demands.
However, the effectiveness of DPO heavily depends on the data quality, which is
frequently compromised by noise. In this work, we propose gamma-PO, a
dynamic target margin preference optimization algorithm that adjust reward
margins at the pairwise level. By introducing instance-specific margin
calibration, gamma-PO strategically prioritizes high-confidence pairs (those
demonstrating higher reward margins) while suppressing potential noise from
ambiguous pairs. Moreover, gamma-PO is a plug-and-play method, compatible
with variants of DPO that rely on reward margin between preference pairs.
Across benchmarks such as AlpacaEval2 and Arena-Hard, gamma-PO achieves an
average 4.4\% improvement over other baselines, setting new benchmarks for
state-of-the-art performance. Additionally, gamma-PO requires minimal code
changes and has a negligible impact on training efficiency, making it a robust
solution for enhancing LLMs alignment. Our codes are available at
https://github.com/sunjie279/gammaPO{https://github.com/sunjie279/gammaPO}.
Qianqi Yan, Hongquan Li, Shan Jiang, Yang Zhao, Xinze Guan, Ching-Chen Kuo, Xin Eric Wang
21
Multimodal large language models (MLLMs) are increasingly deployed in
open-ended, real-world environments where inputs are messy, underspecified, and
not always trustworthy. Unlike curated benchmarks, these settings frequently
involve instructions that refer to missing objects or contradictory facts, rely
on ambiguous references, or request infeasible actions. In such cases, success
hinges not on task execution alone, but on a model's ability to detect when
something is silently wrong. This paper presents a systematic analysis of how
current MLLMs handle such implicit reasoning scenarios: cases where the flaw is
not explicitly stated but must be inferred from context. Using a curated
diagnostic suite spanning four categories of real-world failure modes, we
evaluate six MLLMs, including o3 and GPT-4o, and find that models frequently
fail to surface hidden issues, even when they possess the necessary perceptual
and reasoning skills. Explicit prompting reveals that the underlying
capabilities exist but are often suppressed in favor of user compliance. We
further show that simple inference-time interventions, such as cautious persona
prompting and, in particular, requiring a clarifying question, can dramatically
recover performance. Our findings highlight a persistent gap between reasoning
competence and behavioral compliance in current MLLMs and suggest practical
strategies for making these models more trustworthy in underconstrained
environments.
Xiaorui Wu, Xiaofeng Mao, Xin Zhang, Fei Li, Chong Teng, Yuxiang Peng, Li Zheng, Donghong Ji, Zhuang Li
22
Large language models (LLMs) frequently refuse to respond to pseudo-malicious
instructions: semantically harmless input queries triggering unnecessary LLM
refusals due to conservative safety alignment, significantly impairing user
experience. Collecting such instructions is crucial for evaluating and
mitigating over-refusals, but existing instruction curation methods, like
manual creation or instruction rewriting, either lack scalability or fail to
produce sufficiently diverse and effective refusal-inducing prompts. To address
these limitations, we introduce EVOREFUSE, a prompt optimization approach that
generates diverse pseudo-malicious instructions consistently eliciting
confident refusals across LLMs. EVOREFUSE employs an evolutionary algorithm
exploring the instruction space in more diverse directions than existing
methods via mutation strategies and recombination, and iteratively evolves seed
instructions to maximize evidence lower bound on LLM refusal probability. Using
EVOREFUSE, we create two novel datasets: EVOREFUSE-TEST, a benchmark of 582
pseudo-malicious instructions that outperforms the next-best benchmark with
140.41% higher average refusal triggering rate across 9 LLMs, 34.86% greater
lexical diversity, and 40.03% improved LLM response confidence scores; and
EVOREFUSE-ALIGN, which provides 3,000 pseudo-malicious instructions with
responses for supervised and preference-based alignment training.
LLAMA3.1-8B-INSTRUCT supervisedly fine-tuned on EVOREFUSE-ALIGN achieves up to
14.31% fewer over-refusals than models trained on the second-best alignment
dataset, without compromising safety. Our analysis with EVOREFUSE-TEST reveals
models trigger over-refusals by overly focusing on sensitive keywords while
ignoring broader context.
Maciej Chrabąszcz, Katarzyna Lorenc, Karolina Seweryn
12
Large language models (LLMs) have demonstrated impressive capabilities across
various natural language processing (NLP) tasks in recent years. However, their
susceptibility to jailbreaks and perturbations necessitates additional
evaluations. Many LLMs are multilingual, but safety-related training data
contains mainly high-resource languages like English. This can leave them
vulnerable to perturbations in low-resource languages such as Polish. We show
how surprisingly strong attacks can be cheaply created by altering just a few
characters and using a small proxy model for word importance calculation. We
find that these character and word-level attacks drastically alter the
predictions of different LLMs, suggesting a potential vulnerability that can be
used to circumvent their internal safety mechanisms. We validate our attack
construction methodology on Polish, a low-resource language, and find potential
vulnerabilities of LLMs in this language. Additionally, we show how it can be
extended to other languages. We release the created datasets and code for
further research.
Large Multimodal Models (LMMs) often rely on in-context learning (ICL) to
perform new tasks with minimal supervision. However, ICL performance,
especially in smaller LMMs, is inconsistent and does not always improve
monotonically with increasing examples. We hypothesize that this occurs due to
the LMM being overwhelmed by additional information present in the image
embeddings, which is not required for the downstream task. To address this, we
propose a meta-learning approach that provides an alternative for inducing
few-shot capabilities in LMMs, using a fixed set of soft prompts that are
distilled from task-relevant image features and can be adapted at test time
using a few examples. To facilitate this distillation, we introduce an
attention-mapper module that can be easily integrated with the popular LLaVA
v1.5 architecture and is jointly learned with soft prompts, enabling task
adaptation in LMMs under low-data regimes with just a few gradient steps.
Evaluation on the VL-ICL Bench shows that our method consistently outperforms
ICL and related prompt-tuning approaches, even under image perturbations,
improving task induction and reasoning across visual question answering tasks.