Though reasoning-based large language models (LLMs) have excelled in
mathematics and programming, their capabilities in knowledge-intensive medical
question answering remain underexplored. To address this, we introduce
ReasonMed, the largest medical reasoning dataset, comprising 370k high-quality
examples distilled from 1.7 million initial reasoning paths generated by
various LLMs. ReasonMed is constructed through a multi-agent
verification and refinement process, where we design an Error Refiner
to enhance the reasoning paths by identifying and correcting error-prone steps
flagged by a verifier. Leveraging ReasonMed, we systematically investigate best
practices for training medical reasoning models and find that combining
detailed Chain-of-Thought (CoT) reasoning with concise answer summaries yields
the most effective fine-tuning strategy. Based on this strategy, we train
ReasonMed-7B, which sets a new benchmark for sub-10B models, outperforming the
prior best by 4.17\% and even exceeding LLaMA3.1-70B on PubMedQA by 4.60\%.
Constructing large-scale datasets for the GitHub issue resolution task is
crucial for both training and evaluating the software engineering capabilities
of Large Language Models (LLMs). However, the traditional process for creating
such benchmarks is notoriously challenging and labor-intensive, particularly in
the stages of setting up evaluation environments, grading test outcomes, and
validating task instances. In this paper, we propose SWE-Factory, an automated
pipeline designed to address these challenges. To tackle these issues, our
pipeline integrates three core automated components. First, we introduce
SWE-Builder, a multi-agent system that automates evaluation environment
construction, which employs four specialized agents that work in a
collaborative, iterative loop and leverages an environment memory pool to
enhance efficiency. Second, we introduce a standardized, exit-code-based
grading method that eliminates the need for manually writing custom parsers.
Finally, we automate the fail2pass validation process using these reliable exit
code signals. Experiments on 671 issues across four programming languages show
that our pipeline can effectively construct valid task instances; for example,
with GPT-4.1-mini, our SWE-Builder constructs 269 valid instances at 0.045 per
instance, while with Gemini-2.5-flash, it achieves comparable performance at
the lowest cost of 0.024 per instance. We also demonstrate that our
exit-code-based grading achieves 100% accuracy compared to manual inspection,
and our automated fail2pass validation reaches a precision of 0.92 and a recall
of 1.00. We hope our automated pipeline will accelerate the collection of
large-scale, high-quality GitHub issue resolution datasets for both training
and evaluation. Our code and datasets are released at
https://github.com/DeepSoftwareAnalytics/swe-factory.
Mistral-AI, Abhinav Rastogi, Albert Q. Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, Jason Rute, Joep Barmentlo, Karmesh Yadav, Kartik Khandelwal, Khyathi Raghavi Chandu, Léonard Blier, Lucile Saulnier, Matthieu Dinot, Maxime Darrin, Neha Gupta, Roman Soletskyi, Sagar Vaze, Teven Le Scao, Yihan Wang, Adam Yang, Alexander H. Liu, Alexandre Sablayrolles, Amélie Héliou, Amélie Martin, Andy Ehrenberg, Anmol Agarwal, Antoine Roux, Arthur Darcet, Arthur Mensch, Baptiste Bout, Baptiste Rozière, Baudouin De Monicault, Chris Bamford, Christian Wallenwein, Christophe Renaudin, Clémence Lanfranchi, Darius Dabert, Devon Mizelle, Diego de las Casas, Elliot Chane-Sane, Emilien Fugier, Emma Bou Hanna, Gauthier Delerce, Gauthier Guinet, Georgii Novikov, Guillaume Martin, Himanshu Jaju, Jan Ludziejewski, Jean-Hadrien Chabran, Jean-Malo Delignon, Joachim Studnia, Jonas Amar, Josselin Somerville Roberts, Julien Denize, Karan Saxena, Kush Jain, Lingxiao Zhao, Louis Martin, Luyu Gao, Lélio Renard Lavaud, Marie Pellat, Mathilde Guillaumin, Mathis Felardos, Maximilian Augustin, Mickaël Seznec, Nikhil Raghuraman, Olivier Duchenne, Patricia Wang, Patrick von Platen, Patryk Saffer, Paul Jacob, Paul Wambergue, Paula Kurylowicz, Pavankumar Reddy Muddireddy, Philomène Chagniot, Pierre Stock, Pravesh Agrawal, Romain Sauvestre, Rémi Delacourt, Sanchit Gandhi, Sandeep Subramanian, Shashwat Dalal, Siddharth Gandhi, Soham Ghosh, Srijan Mishra, Sumukh Aithal, Szymon Antoniak, Thibault Schueller, Thibaut Lavril, Thomas Robert, Thomas Wang, Timothée Lacroix, Valeriia Nemychnikova, Victor Paltz, Virgile Richard, Wen-Ding Li, William Marshall, Xuanyu Zhang, Yunhao Tang
353
We introduce Magistral, Mistral's first reasoning model and our own scalable
reinforcement learning (RL) pipeline. Instead of relying on existing
implementations and RL traces distilled from prior models, we follow a ground
up approach, relying solely on our own models and infrastructure. Notably, we
demonstrate a stack that enabled us to explore the limits of pure RL training
of LLMs, present a simple method to force the reasoning language of the model,
and show that RL on text data alone maintains most of the initial checkpoint's
capabilities. We find that RL on text maintains or improves multimodal
understanding, instruction following and function calling. We present Magistral
Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we
open-source Magistral Small (Apache 2.0) which further includes cold-start data
from Magistral Medium.
Jaewon Min, Jin Hyeon Kim, Paul Hyunbin Cho, Jaeeun Lee, Jihye Park, Minkyu Park, Sangpil Kim, Hyunhee Park, Seungryong Kim
352
Image restoration aims to recover degraded images. However, existing
diffusion-based restoration methods, despite great success in natural image
restoration, often struggle to faithfully reconstruct textual regions in
degraded images. Those methods frequently generate plausible but incorrect
text-like patterns, a phenomenon we refer to as text-image hallucination. In
this paper, we introduce Text-Aware Image Restoration (TAIR), a novel
restoration task that requires the simultaneous recovery of visual contents and
textual fidelity. To tackle this task, we present SA-Text, a large-scale
benchmark of 100K high-quality scene images densely annotated with diverse and
complex text instances. Furthermore, we propose a multi-task diffusion
framework, called TeReDiff, that integrates internal features from diffusion
models into a text-spotting module, enabling both components to benefit from
joint training. This allows for the extraction of rich text representations,
which are utilized as prompts in subsequent denoising steps. Extensive
experiments demonstrate that our approach consistently outperforms
state-of-the-art restoration methods, achieving significant gains in text
recognition accuracy. See our project page: https://cvlab-kaist.github.io/TAIR/
Despite rapid advancements in video generation models, generating coherent
storytelling videos that span multiple scenes and characters remains
challenging. Current methods often rigidly convert pre-generated keyframes into
fixed-length clips, resulting in disjointed narratives and pacing issues.
Furthermore, the inherent instability of video generation models means that
even a single low-quality clip can significantly degrade the entire output
animation's logical coherence and visual continuity. To overcome these
obstacles, we introduce AniMaker, a multi-agent framework enabling efficient
multi-candidate clip generation and storytelling-aware clip selection, thus
creating globally consistent and story-coherent animation solely from text
input. The framework is structured around specialized agents, including the
Director Agent for storyboard generation, the Photography Agent for video clip
generation, the Reviewer Agent for evaluation, and the Post-Production Agent
for editing and voiceover. Central to AniMaker's approach are two key technical
components: MCTS-Gen in Photography Agent, an efficient Monte Carlo Tree Search
(MCTS)-inspired strategy that intelligently navigates the candidate space to
generate high-potential clips while optimizing resource usage; and AniEval in
Reviewer Agent, the first framework specifically designed for multi-shot
animation evaluation, which assesses critical aspects such as story-level
consistency, action completion, and animation-specific features by considering
each clip in the context of its preceding and succeeding clips. Experiments
demonstrate that AniMaker achieves superior quality as measured by popular
metrics including VBench and our proposed AniEval framework, while
significantly improving the efficiency of multi-candidate generation, pushing
AI-generated storytelling animation closer to production standards.
We present VRBench, the first long narrative video benchmark crafted for
evaluating large models' multi-step reasoning capabilities, addressing
limitations in existing evaluations that overlook temporal reasoning and
procedural validity. It comprises 1,010 long videos (with an average duration
of 1.6 hours), along with 9,468 human-labeled multi-step question-answering
pairs and 30,292 reasoning steps with timestamps. These videos are curated via
a multi-stage filtering process including expert inter-rater reviewing to
prioritize plot coherence. We develop a human-AI collaborative framework that
generates coherent reasoning chains, each requiring multiple temporally
grounded steps, spanning seven types (e.g., event attribution, implicit
inference). VRBench designs a multi-phase evaluation pipeline that assesses
models at both the outcome and process levels. Apart from the MCQs for the
final results, we propose a progress-level LLM-guided scoring metric to
evaluate the quality of the reasoning chain from multiple dimensions
comprehensively. Through extensive evaluations of 12 LLMs and 16 VLMs on
VRBench, we undertake a thorough analysis and provide valuable insights that
advance the field of multi-step reasoning.
Pooneh Mousavi, Gallil Maimon, Adel Moumen, Darius Petermann, Jiatong Shi, Haibin Wu, Haici Yang, Anastasia Kuznetsova, Artem Ploujnikov, Ricard Marxer, Bhuvana Ramabhadran, Benjamin Elizalde, Loren Lugosch, Jinyu Li, Cem Subakan, Phil Woodland, Minje Kim, Hung-yi Lee, Shinji Watanabe, Yossi Adi, Mirco Ravanelli
242
Discrete audio tokens are compact representations that aim to preserve
perceptual quality, phonetic content, and speaker characteristics while
enabling efficient storage and inference, as well as competitive performance
across diverse downstream tasks.They provide a practical alternative to
continuous features, enabling the integration of speech and audio into modern
large language models (LLMs). As interest in token-based audio processing
grows, various tokenization methods have emerged, and several surveys have
reviewed the latest progress in the field. However, existing studies often
focus on specific domains or tasks and lack a unified comparison across various
benchmarks. This paper presents a systematic review and benchmark of discrete
audio tokenizers, covering three domains: speech, music, and general audio. We
propose a taxonomy of tokenization approaches based on encoder-decoder,
quantization techniques, training paradigm, streamability, and application
domains. We evaluate tokenizers on multiple benchmarks for reconstruction,
downstream performance, and acoustic language modeling, and analyze trade-offs
through controlled ablation studies. Our findings highlight key limitations,
practical considerations, and open challenges, providing insight and guidance
for future research in this rapidly evolving area. For more information,
including our main results and tokenizer database, please refer to our website:
https://poonehmousavi.github.io/dates-website/.
Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Minjae Kim, Jaewon Min, Wooseok Jang, Saungwu Lee, Sayak Paul, Susung Hong, Seungryong Kim
203
Recent guidance methods in diffusion models steer reverse sampling by
perturbing the model to construct an implicit weak model and guide generation
away from it. Among these approaches, attention perturbation has demonstrated
strong empirical performance in unconditional scenarios where classifier-free
guidance is not applicable. However, existing attention perturbation methods
lack principled approaches for determining where perturbations should be
applied, particularly in Diffusion Transformer (DiT) architectures where
quality-relevant computations are distributed across layers. In this paper, we
investigate the granularity of attention perturbations, ranging from the layer
level down to individual attention heads, and discover that specific heads
govern distinct visual concepts such as structure, style, and texture quality.
Building on this insight, we propose "HeadHunter", a systematic framework for
iteratively selecting attention heads that align with user-centric objectives,
enabling fine-grained control over generation quality and visual attributes. In
addition, we introduce SoftPAG, which linearly interpolates each selected
head's attention map toward an identity matrix, providing a continuous knob to
tune perturbation strength and suppress artifacts. Our approach not only
mitigates the oversmoothing issues of existing layer-level perturbation but
also enables targeted manipulation of specific visual styles through
compositional head selection. We validate our method on modern large-scale
DiT-based text-to-image models including Stable Diffusion 3 and FLUX.1,
demonstrating superior performance in both general quality enhancement and
style-specific guidance. Our work provides the first head-level analysis of
attention perturbation in diffusion models, uncovering interpretable
specialization within attention layers and enabling practical design of
effective perturbation strategies.
We introduce~Domain2Vec, a novel approach that decomposes any
dataset into a linear combination of several meta-domains, a new concept
designed to capture the key underlying features of datasets.
Domain2Vec maintains a vocabulary of meta-domains and uses a
classifier to decompose any given dataset into a domain vector that corresponds
to a distribution over this vocabulary. These domain vectors enable the
identification of the optimal data mixture for language model (LM) pretraining
in a training-free manner under the \textbf{Distribution
Alignment Assumption} (DA^{2}), which suggests that when
the data distributions of the training set and the validation set are better
aligned, a lower validation loss is achieved. Moreover, Domain2vec can
be seamlessly integrated into previous works to model the relationship between
domain vectors and LM performance, greatly enhancing the efficiency and
scalability of previous methods. Extensive experiments demonstrate that
Domain2Vec helps find the data mixture that enhances downstream task
performance with minimal computational overhead. Specifically,
Domain2Vec achieves the same validation loss on Pile-CC using only
51.5% of the computation required when training on the original mixture of
The Pile dataset. Under equivalent compute budget, Domain2Vec improves
downstream performance by an average of 2.83%.
We propose Ming-Omni, a unified multimodal model capable of processing
images, text, audio, and video, while demonstrating strong proficiency in both
speech and image generation. Ming-Omni employs dedicated encoders to extract
tokens from different modalities, which are then processed by Ling, an MoE
architecture equipped with newly proposed modality-specific routers. This
design enables a single model to efficiently process and fuse multimodal inputs
within a unified framework, thereby facilitating diverse tasks without
requiring separate models, task-specific fine-tuning, or structural redesign.
Importantly, Ming-Omni extends beyond conventional multimodal models by
supporting audio and image generation. This is achieved through the integration
of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for
high-quality image generation, which also allow the model to engage in
context-aware chatting, perform text-to-speech conversion, and conduct
versatile image editing. Our experimental results showcase Ming-Omni offers a
powerful solution for unified perception and generation across all modalities.
Notably, our proposed Ming-Omni is the first open-source model we are aware of
to match GPT-4o in modality support, and we release all code and model weights
to encourage further research and development in the community.
SiXiang Chen, Jianyu Lai, Jialin Gao, Tian Ye, Haoyu Chen, Hengyu Shi, Shitong Shao, Yunlong Lin, Song Fei, Zhaohu Xing, Yeying Jin, Junfeng Luo, Xiaoming Wei, Lei Zhu
193
Generating aesthetic posters is more challenging than simple design images:
it requires not only precise text rendering but also the seamless integration
of abstract artistic content, striking layouts, and overall stylistic harmony.
To address this, we propose PosterCraft, a unified framework that abandons
prior modular pipelines and rigid, predefined layouts, allowing the model to
freely explore coherent, visually compelling compositions. PosterCraft employs
a carefully designed, cascaded workflow to optimize the generation of
high-aesthetic posters: (i) large-scale text-rendering optimization on our
newly introduced Text-Render-2M dataset; (ii) region-aware supervised
fine-tuning on HQ-Poster100K; (iii) aesthetic-text-reinforcement learning via
best-of-n preference optimization; and (iv) joint vision-language feedback
refinement. Each stage is supported by a fully automated data-construction
pipeline tailored to its specific needs, enabling robust training without
complex architectural modifications. Evaluated on multiple experiments,
PosterCraft significantly outperforms open-source baselines in rendering
accuracy, layout coherence, and overall visual appeal-approaching the quality
of SOTA commercial systems. Our code, models, and datasets can be found in the
Project page: https://ephemeral182.github.io/PosterCraft
Recently, agents based on multimodal large language models (MLLMs) have
achieved remarkable progress across various domains. However, building a
generalist agent with capabilities such as perception, planning, action,
grounding, and reflection in open-world environments like Minecraft remains
challenges: insufficient domain-specific data, interference among heterogeneous
tasks, and visual diversity in open-world settings. In this paper, we address
these challenges through three key contributions. 1) We propose a
knowledge-enhanced data generation pipeline to provide scalable and
high-quality training data for agent development. 2) To mitigate interference
among heterogeneous tasks, we introduce a Mixture-of-Experts (MoE) architecture
with task-level routing. 3) We develop a Multimodal Reasoning-Augmented
Reinforcement Learning approach to enhance the agent's reasoning ability for
visual diversity in Minecraft. Built upon these innovations, we present
Optimus-3, a general-purpose agent for Minecraft. Extensive experimental
results demonstrate that Optimus-3 surpasses both generalist multimodal large
language models and existing state-of-the-art agents across a wide range of
tasks in the Minecraft environment. Project page:
https://cybertronagent.github.io/Optimus-3.github.io/
How cost-effectively can we elicit strong reasoning in language models by
leveraging their underlying representations? We answer this question with Resa,
a family of 1.5B reasoning models trained via a novel and efficient sparse
autoencoder tuning (SAE-Tuning) procedure. This method first trains an SAE to
capture reasoning abilities from a source model, and then uses the trained SAE
to guide a standard supervised fine-tuning process to elicit such abilities in
a target model, all using verified question-answer data without any reasoning
traces. Notably, when applied to certain base models before further RL
post-training, SAE-Tuning retains >97% of its RL-trained counterpart's
reasoning performance while reducing training costs by >2000x to roughly \1
and training time by >450x to around 20 minutes. Furthermore, when applied to
lightly RL-trained models (e.g., within 1 hour on 2 GPUs), it enables reasoning
performance such as 43.33% Pass@1 on AIME24 and 90% Pass@1 on AMC23 for only
around 1 additional cost. Surprisingly, the reasoning abilities extracted via
SAEs are potentially both generalizable and modular. Generality means abilities
extracted from one dataset still elevate performance on a larger and
overlapping corpus. Modularity means abilities extracted from Qwen or Qwen-Math
can be attached to the R1-Distill model at test time, without any retraining,
and yield comparable gains. Extensive ablations validate these findings and all
artifacts are fully open-sourced.
Shojaee et al. (2025) report that Large Reasoning Models (LRMs) exhibit
"accuracy collapse" on planning puzzles beyond certain complexity thresholds.
We demonstrate that their findings primarily reflect experimental design
limitations rather than fundamental reasoning failures. Our analysis reveals
three critical issues: (1) Tower of Hanoi experiments systematically exceed
model output token limits at reported failure points, with models explicitly
acknowledging these constraints in their outputs; (2) The authors' automated
evaluation framework fails to distinguish between reasoning failures and
practical constraints, leading to misclassification of model capabilities; (3)
Most concerningly, their River Crossing benchmarks include mathematically
impossible instances for N > 5 due to insufficient boat capacity, yet models
are scored as failures for not solving these unsolvable problems. When we
control for these experimental artifacts, by requesting generating functions
instead of exhaustive move lists, preliminary experiments across multiple
models indicate high accuracy on Tower of Hanoi instances previously reported
as complete failures. These findings highlight the importance of careful
experimental design when evaluating AI reasoning capabilities.
Long video understanding (LVU) presents a significant challenge for current
multi-modal large language models (MLLMs) due to the task's inherent complexity
and context window constraint. It is widely assumed that addressing LVU tasks
requires foundation MLLMs with extended context windows, strong visual
perception capabilities, and proficient domain expertise. In this work, we
challenge this common belief by introducing VideoDeepResearch, a novel agentic
framework for long video understanding. Our approach relies solely on a
text-only large reasoning model (LRM) combined with a modular multi-modal
toolkit, including multimodal retrievers and visual perceivers, all of which
are readily available in practice. For each LVU task, the system formulates a
problem-solving strategy through reasoning, while selectively accessing and
utilizing essential video content via tool using. We conduct extensive
experiments on popular LVU benchmarks, including MLVU, Video-MME, and LVBench.
Our results demonstrate that VideoDeepResearch achieves substantial
improvements over existing MLLM baselines, surpassing the previous
state-of-the-art by 9.6%, 6.6%, and 3.9% on MLVU (test), LVBench, and
LongVideoBench, respectively. These findings highlight the promise of agentic
systems in overcoming key challenges in LVU problems.
Large Language Model (LLM) agents have shown great potential in addressing
real-world data science problems. LLM-driven data science agents promise to
automate the entire machine learning pipeline, yet their real-world
effectiveness remains limited. Existing frameworks depend on rigid, pre-defined
workflows and inflexible coding strategies; consequently, they excel only on
relatively simple, classical problems and fail to capture the empirical
expertise that human practitioners bring to complex, innovative tasks. In this
work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework
that overcomes these deficiencies through three key advances: (1) a curated
expert knowledge base that grounds the agent in domain expert knowledge, (2) an
agentic knowledgeable tree search algorithm that strategically explores
possible solutions, and (3) a self-adaptive coding strategy that dynamically
tailors code generation to task complexity. Evaluations on two automated data
science benchmarks demonstrate that AutoMind delivers superior performance
versus state-of-the-art baselines. Additional analyses confirm favorable
effectiveness, efficiency, and qualitative solution quality, highlighting
AutoMind as an efficient and robust step toward fully automated data science.
Xing Han Lù, Gaurav Kamath, Marius Mosbach, Siva Reddy
142
Recent advancements in Large Language Models (LLMs) and multimodal
counterparts have spurred significant interest in developing web agents -- AI
systems capable of autonomously navigating and completing tasks within web
environments. While holding tremendous promise for automating complex web
interactions, current approaches face substantial challenges due to the
fundamental mismatch between human-designed interfaces and LLM capabilities.
Current methods struggle with the inherent complexity of web inputs, whether
processing massive DOM trees, relying on screenshots augmented with additional
information, or bypassing the user interface entirely through API interactions.
This position paper advocates for a paradigm shift in web agent research:
rather than forcing web agents to adapt to interfaces designed for humans, we
should develop a new interaction paradigm specifically optimized for agentic
capabilities. To this end, we introduce the concept of an Agentic Web Interface
(AWI), an interface specifically designed for agents to navigate a website. We
establish six guiding principles for AWI design, emphasizing safety,
efficiency, and standardization, to account for the interests of all primary
stakeholders. This reframing aims to overcome fundamental limitations of
existing interfaces, paving the way for more efficient, reliable, and
transparent web agent design, which will be a collaborative effort involving
the broader ML community.
Zhao Zhang, Yutao Cheng, Dexiang Hong, Maoke Yang, Gonglei Shi, Lei Ma, Hui Zhang, Jie Shao, Xinglong Wu
102
Graphic design plays a crucial role in both commercial and personal contexts,
yet creating high-quality, editable, and aesthetically pleasing graphic
compositions remains a time-consuming and skill-intensive task, especially for
beginners. Current AI tools automate parts of the workflow, but struggle to
accurately incorporate user-supplied assets, maintain editability, and achieve
professional visual appeal. Commercial systems, like Canva Magic Design, rely
on vast template libraries, which are impractical for replicate. In this paper,
we introduce CreatiPoster, a framework that generates editable, multi-layer
compositions from optional natural-language instructions or assets. A protocol
model, an RGBA large multimodal model, first produces a JSON specification
detailing every layer (text or asset) with precise layout, hierarchy, content
and style, plus a concise background prompt. A conditional background model
then synthesizes a coherent background conditioned on this rendered foreground
layers. We construct a benchmark with automated metrics for graphic-design
generation and show that CreatiPoster surpasses leading open-source approaches
and proprietary commercial systems. To catalyze further research, we release a
copyright-free corpus of 100,000 multi-layer designs. CreatiPoster supports
diverse applications such as canvas editing, text overlay, responsive resizing,
multilingual adaptation, and animated posters, advancing the democratization of
AI-assisted graphic design. Project homepage:
https://github.com/graphic-design-ai/creatiposter
Kangwei Liu, Siyuan Cheng, Bozhong Tian, Xiaozhuan Liang, Yuyang Yin, Meng Han, Ningyu Zhang, Bryan Hooi, Xi Chen, Shumin Deng
92
Large language models (LLMs) have been increasingly applied to automated
harmful content detection tasks, assisting moderators in identifying policy
violations and improving the overall efficiency and accuracy of content review.
However, existing resources for harmful content detection are predominantly
focused on English, with Chinese datasets remaining scarce and often limited in
scope. We present a comprehensive, professionally annotated benchmark for
Chinese content harm detection, which covers six representative categories and
is constructed entirely from real-world data. Our annotation process further
yields a knowledge rule base that provides explicit expert knowledge to assist
LLMs in Chinese harmful content detection. In addition, we propose a
knowledge-augmented baseline that integrates both human-annotated knowledge
rules and implicit knowledge from large language models, enabling smaller
models to achieve performance comparable to state-of-the-art LLMs. Code and
data are available at https://github.com/zjunlp/ChineseHarm-bench.
Ying Shen, Zhiyang Xu, Jiuhai Chen, Shizhe Diao, Jiaxin Zhang, Yuguang Yao, Joy Rimchala, Ismini Lourentzou, Lifu Huang
92
Recent advances in multimodal foundation models unifying image understanding
and generation have opened exciting avenues for tackling a wide range of
vision-language tasks within a single framework. Despite progress, existing
unified models typically require extensive pretraining and struggle to achieve
the same level of performance compared to models dedicated to each task.
Additionally, many of these models suffer from slow image generation speeds,
limiting their practical deployment in real-time or resource-constrained
settings. In this work, we propose Layerwise Timestep-Expert Flow-based
Transformer (LaTtE-Flow), a novel and efficient architecture that unifies image
understanding and generation within a single multimodal model. LaTtE-Flow
builds upon powerful pretrained Vision-Language Models (VLMs) to inherit strong
multimodal understanding capabilities, and extends them with a novel Layerwise
Timestep Experts flow-based architecture for efficient image generation.
LaTtE-Flow distributes the flow-matching process across specialized groups of
Transformer layers, each responsible for a distinct subset of timesteps. This
design significantly improves sampling efficiency by activating only a small
subset of layers at each sampling timestep. To further enhance performance, we
propose a Timestep-Conditioned Residual Attention mechanism for efficient
information reuse across layers. Experiments demonstrate that LaTtE-Flow
achieves strong performance on multimodal understanding tasks, while achieving
competitive image generation quality with around 6x faster inference speed
compared to recent unified multimodal models.
Bill Psomas, Dionysis Christopoulos, Eirini Baltzi, Ioannis Kakogeorgiou, Tilemachos Aravanis, Nikos Komodakis, Konstantinos Karantzalos, Yannis Avrithis, Giorgos Tolias
72
As fine-tuning (FT) becomes increasingly impractical at scale, probing is
emerging as the preferred evaluation protocol for self-supervised learning
(SSL). Yet, the standard linear probing (LP) fails to adequately reflect the
potential of models trained with Masked Image Modeling (MIM), due to the
distributed nature of patch tokens. This motivates the need for attentive
probing, an alternative that uses attention to selectively aggregate
patch-level features. Despite its growing adoption, attentive probing remains
under-explored, with existing methods suffering from excessive parameterization
and poor computational efficiency.
In this work, we revisit attentive probing through the lens of the
accuracy-efficiency trade-off. We conduct a systematic study of existing
methods, analyzing their mechanisms and benchmarking their performance. We
introduce efficient probing (EP), a multi-query cross-attention mechanism that
eliminates redundant projections, reduces the number of trainable parameters,
and achieves up to a 10times speed-up over conventional multi-head
attention. Despite its simplicity, EP outperforms LP and prior attentive
probing approaches across seven benchmarks, generalizes well beyond MIM to
diverse pre-training paradigms, produces interpretable attention maps, and
achieves strong gains in low-shot and layer-wise settings. Code available at
https://github.com/billpsomas/efficient-probing.
Recent advancements in large language models (LLMs) and AI systems have led
to a paradigm shift in the design and optimization of complex AI workflows. By
integrating multiple components, compound AI systems have become increasingly
adept at performing sophisticated tasks. However, as these systems grow in
complexity, new challenges arise in optimizing not only individual components
but also their interactions. While traditional optimization methods such as
supervised fine-tuning (SFT) and reinforcement learning (RL) remain
foundational, the rise of natural language feedback introduces promising new
approaches, especially for optimizing non-differentiable systems. This paper
provides a systematic review of recent progress in optimizing compound AI
systems, encompassing both numerical and language-based techniques. We
formalize the notion of compound AI system optimization, classify existing
methods along several key dimensions, and highlight open research challenges
and future directions in this rapidly evolving field. A list of surveyed papers
is publicly available at https://github.com/MiuLab/AISysOpt-Survey.
Do Xuan Long, Duy Dinh, Ngoc-Hai Nguyen, Kenji Kawaguchi, Nancy F. Chen, Shafiq Joty, Min-Yen Kan
62
As large language models (LLMs) have progressed towards more human-like and
human--AI communications have become prevalent, prompting has emerged as a
decisive component. However, there is limited conceptual consensus on what
exactly quantifies natural language prompts. We attempt to address this
question by conducting a meta-analysis surveying more than 150
prompting-related papers from leading NLP and AI conferences from 2022 to 2025
and blogs. We propose a property- and human-centric framework for evaluating
prompt quality, encompassing 21 properties categorized into six dimensions. We
then examine how existing studies assess their impact on LLMs, revealing their
imbalanced support across models and tasks, and substantial research gaps.
Further, we analyze correlations among properties in high-quality natural
language prompts, deriving prompting recommendations. We then empirically
explore multi-property prompt enhancements in reasoning tasks, observing that
single-property enhancements often have the greatest impact. Finally, we
discover that instruction-tuning on property-enhanced prompts can result in
better reasoning models. Our findings establish a foundation for
property-centric prompt evaluation and optimization, bridging the gaps between
human--AI communication and opening new prompting research directions.
The scale diversity of point cloud data presents significant challenges in
developing unified representation learning techniques for 3D vision. Currently,
there are few unified 3D models, and no existing pre-training method is equally
effective for both object- and scene-level point clouds. In this paper, we
introduce UniPre3D, the first unified pre-training method that can be
seamlessly applied to point clouds of any scale and 3D models of any
architecture. Our approach predicts Gaussian primitives as the pre-training
task and employs differentiable Gaussian splatting to render images, enabling
precise pixel-level supervision and end-to-end optimization. To further
regulate the complexity of the pre-training task and direct the model's focus
toward geometric structures, we integrate 2D features from pre-trained image
models to incorporate well-established texture knowledge. We validate the
universal effectiveness of our proposed method through extensive experiments
across a variety of object- and scene-level tasks, using diverse point cloud
models as backbones. Code is available at https://github.com/wangzy22/UniPre3D.
Hao Peng, Yunjia Qi, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
52
Reinforcement learning with verifiable rewards (RLVR) has become a key
technique for enhancing large language models (LLMs), with verification
engineering playing a central role. However, best practices for RL in
instruction following remain underexplored. In this work, we explore the
verification challenge in RL for instruction following and propose VerIF, a
verification method that combines rule-based code verification with LLM-based
verification from a large reasoning model (e.g., QwQ-32B). To support this
approach, we construct a high-quality instruction-following dataset,
VerInstruct, containing approximately 22,000 instances with associated
verification signals. We apply RL training with VerIF to two models, achieving
significant improvements across several representative instruction-following
benchmarks. The trained models reach state-of-the-art performance among models
of comparable size and generalize well to unseen constraints. We further
observe that their general capabilities remain unaffected, suggesting that RL
with VerIF can be integrated into existing RL recipes to enhance overall model
performance. We have released our datasets, codes, and models to facilitate
future research at https://github.com/THU-KEG/VerIF.
Large language models have transformed natural language processing, yet
supervised fine-tuning (SFT) remains computationally intensive. This paper
formally proves that capabilities acquired through SFT can be approximated by a
base transformer model using inference-time techniques, specifically in-context
learning (ICL), without altering model parameters, under idealized assumptions
including unbounded computational resources and access to the fine-tuning
dataset. We extend these results to practical scenarios with finite context
lengths and partial dataset access. For text generation tasks with fixed output
length l, datasets of size Oleft( m V{varepsilon^2} log
m{delta} right) or, with bounded context, Oleft( l
log V{varepsilon^2} log 1{delta} right) suffice to approximate
fine-tuned behavior across m contexts within error varepsilon, where V
is the vocabulary size and delta is the failure probability. For linear
classification, datasets of size Oleft( d{varepsilon}
right) or, with fixed context, Oleft( 1{varepsilon^2} log
1{delta} right) are sufficient, where d is the input dimension.
Grounded in the Turing completeness of transformers, these results provide a
theoretical foundation for resource-efficient deployment of large language
models, with practical techniques like retrieval-augmented generation bridging
theory to real-world applications.
A central goal for mechanistic interpretability has been to identify the
right units of analysis in large language models (LLMs) that causally explain
their outputs. While early work focused on individual neurons, evidence that
neurons often encode multiple concepts has motivated a shift toward analyzing
directions in activation space. A key question is how to find directions that
capture interpretable features in an unsupervised manner. Current methods rely
on dictionary learning with sparse autoencoders (SAEs), commonly trained over
residual stream activations to learn directions from scratch. However, SAEs
often struggle in causal evaluations and lack intrinsic interpretability, as
their learning is not explicitly tied to the computations of the model. Here,
we tackle these limitations by directly decomposing MLP activations with
semi-nonnegative matrix factorization (SNMF), such that the learned features
are (a) sparse linear combinations of co-activated neurons, and (b) mapped to
their activating inputs, making them directly interpretable. Experiments on
Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs
and a strong supervised baseline (difference-in-means) on causal steering,
while aligning with human-interpretable concepts. Further analysis reveals that
specific neuron combinations are reused across semantically-related features,
exposing a hierarchical structure in the MLP's activation space. Together,
these results position SNMF as a simple and effective tool for identifying
interpretable features and dissecting concept representations in LLMs.
Jari Kolehmainen, Nikolay Blagoev, John Donaghy, Oğuzhan Ersoy, Christopher Nies
42
Training large language models is generally done via optimization methods on
clusters containing tens of thousands of accelerators, communicating over a
high-bandwidth interconnect. Scaling up these clusters is expensive and can
become impractical, imposing limits on the size of models that can be trained.
Several recent studies have proposed training methods that are less
communication intensive, avoiding the need for a highly connected compute
cluster. These state-of-the-art low communication training methods still employ
a synchronization step for model parameters, which, when performed over all
model replicas, can become costly on a low-bandwidth network.
In this work, we propose a novel optimization method, NoLoCo, that does not
explicitly synchronize all model parameters during training and, as a result,
does not require any collective communication. NoLoCo implicitly synchronizes
model weights via a novel variant of the Nesterov momentum optimizer by
partially averaging model weights with a randomly selected other one. We
provide both a theoretical convergence analysis for our proposed optimizer as
well as empirical results from language model training.
We benchmark NoLoCo on a wide range of accelerator counts and model sizes,
between 125M to 6.8B parameters. Our method requires significantly less
communication overhead than fully sharded data parallel training or even widely
used low communication training method, DiLoCo. The synchronization step itself
is estimated to be one magnitude faster than the all-reduce used in DiLoCo for
few hundred accelerators training over the internet. We also do not have any
global blocking communication that reduces accelerator idling time. Compared to
DiLoCo, we also observe up to 4% faster convergence rate with wide range of
model sizes and accelerator counts.
In e-commerce and digital marketing, generating high-fidelity human-product
demonstration videos is important for effective product presentation. However,
most existing frameworks either fail to preserve the identities of both humans
and products or lack an understanding of human-product spatial relationships,
leading to unrealistic representations and unnatural interactions. To address
these challenges, we propose a Diffusion Transformer (DiT)-based framework. Our
method simultaneously preserves human identities and product-specific details,
such as logos and textures, by injecting paired human-product reference
information and utilizing an additional masked cross-attention mechanism. We
employ a 3D body mesh template and product bounding boxes to provide precise
motion guidance, enabling intuitive alignment of hand gestures with product
placements. Additionally, structured text encoding is used to incorporate
category-level semantics, enhancing 3D consistency during small rotational
changes across frames. Trained on a hybrid dataset with extensive data
augmentation strategies, our approach outperforms state-of-the-art techniques
in maintaining the identity integrity of both humans and products and
generating realistic demonstration motions. Project page:
https://submit2025-dream.github.io/DreamActor-H1/.
Classifier-free guidance (CFG) has become an essential component of modern
diffusion models to enhance both generation quality and alignment with input
conditions. However, CFG requires specific training procedures and is limited
to conditional generation. To address these limitations, we propose Token
Perturbation Guidance (TPG), a novel method that applies perturbation matrices
directly to intermediate token representations within the diffusion network.
TPG employs a norm-preserving shuffling operation to provide effective and
stable guidance signals that improve generation quality without architectural
changes. As a result, TPG is training-free and agnostic to input conditions,
making it readily applicable to both conditional and unconditional generation.
We further analyze the guidance term provided by TPG and show that its effect
on sampling more closely resembles CFG compared to existing training-free
guidance techniques. Extensive experiments on SDXL and Stable Diffusion 2.1
show that TPG achieves nearly a 2times improvement in FID for unconditional
generation over the SDXL baseline, while closely matching CFG in prompt
alignment. These results establish TPG as a general, condition-agnostic
guidance method that brings CFG-like benefits to a broader class of diffusion
models. The code is available at
https://github.com/TaatiTeam/Token-Perturbation-Guidance
Vincenzo Colle, Mohamed Sana, Nicola Piovesan, Antonio De Domenico, Fadhel Ayed, Merouane Debbah
32
The increasing adoption of artificial intelligence in telecommunications has
raised interest in the capability of Large Language Models (LLMs) to address
domain-specific, mathematically intensive tasks. Although recent advancements
have improved the performance of LLMs in general mathematical reasoning, their
effectiveness within specialized domains, such as signal processing, network
optimization, and performance analysis, remains largely unexplored. To address
this gap, we introduce TeleMath, the first benchmark dataset specifically
designed to evaluate LLM performance in solving mathematical problems with
numerical solutions in the telecommunications domain. Comprising 500
question-answer (QnA) pairs, TeleMath covers a wide spectrum of topics in the
telecommunications field. This paper outlines the proposed QnAs generation
pipeline, starting from a selected seed of problems crafted by Subject Matter
Experts. The evaluation of a wide range of open-source LLMs reveals that best
performance on TeleMath is achieved by recent models explicitly designed for
mathematical or logical reasoning. In contrast, general-purpose models, even
those with a large number of parameters, often struggle with these challenges.
We have released the dataset and the evaluation code to ease result
reproducibility and support future research.
Kevin Galim, Ethan Ewer, Wonjun Kang, Minjae Lee, Hyung Il Koo, Kangwook Lee
32
Optimizing inference for long-context Large Language Models (LLMs) is
increasingly important due to the quadratic compute and linear memory
complexity of Transformers. Existing approximation methods, such as key-value
(KV) cache dropping, sparse attention, and prompt compression, typically rely
on rough predictions of token or KV pair importance. We propose a novel
framework for approximate LLM inference that leverages small draft models to
more accurately predict the importance of tokens and KV pairs. Specifically, we
introduce two instantiations of our proposed framework: (i) SpecKV, which
leverages a draft output to accurately assess the importance of each KV pair
for more effective KV cache dropping, and (ii) SpecPC, which uses the draft
model's attention activations to identify and discard unimportant prompt
tokens. To the best of our knowledge, this is the first work to use draft
models for approximate LLM inference acceleration, extending their utility
beyond traditional lossless speculative decoding. We motivate our methods with
theoretical and empirical analyses, and show a strong correlation between the
attention patterns of draft and target models. Extensive experiments on
long-context benchmarks show that our methods consistently achieve higher
accuracy than existing baselines, while preserving the same improvements in
memory usage, latency, and throughput. Our code is available at
https://github.com/furiosa-ai/draft-based-approx-llm.
Large Language Model (LLM) unlearning aims to erase or suppress undesirable
knowledge within the model, offering promise for controlling harmful or private
information to prevent misuse. However, recent studies highlight its limited
efficacy in real-world scenarios, hindering practical adoption. In this study,
we identify a pervasive issue underlying many downstream failures: the
effectiveness of existing unlearning methods heavily depends on the form of
training samples and frequently fails to generalize to alternate expressions of
the same knowledge. We formally characterize this problem as Form-Dependent
Bias and systematically investigate its specific manifestation patterns across
various downstream tasks. To quantify its prevalence and support future
research, we introduce ORT, a novel benchmark designed to evaluate the
robustness of unlearning methods against variations in knowledge expression.
Results reveal that Form-Dependent Bias is both widespread and severe among
current techniques.
We argue that LLM unlearning should be form-independent to address the
endless forms of downstream tasks encountered in real-world security-critical
scenarios. Towards this goal, we introduce Rank-one Concept Redirection (ROCR),
a novel training-free method, as a promising solution path. ROCR performs
unlearning by targeting the invariants in downstream tasks, specifically the
activated dangerous concepts. It is capable of modifying model parameters
within seconds to redirect the model's perception of a specific unlearning
target concept to another harmless concept. Extensive experiments demonstrate
that ROCR significantly improves unlearning effectiveness compared to
traditional methods while generating highly natural outputs.
Priyanka Kargupta, Nan Zhang, Yunyi Zhang, Rui Zhang, Prasenjit Mitra, Jiawei Han
22
The rapid evolution of scientific fields introduces challenges in organizing
and retrieving scientific literature. While expert-curated taxonomies have
traditionally addressed this need, the process is time-consuming and expensive.
Furthermore, recent automatic taxonomy construction methods either (1)
over-rely on a specific corpus, sacrificing generalizability, or (2) depend
heavily on the general knowledge of large language models (LLMs) contained
within their pre-training datasets, often overlooking the dynamic nature of
evolving scientific domains. Additionally, these approaches fail to account for
the multi-faceted nature of scientific literature, where a single research
paper may contribute to multiple dimensions (e.g., methodology, new tasks,
evaluation metrics, benchmarks). To address these gaps, we propose TaxoAdapt, a
framework that dynamically adapts an LLM-generated taxonomy to a given corpus
across multiple dimensions. TaxoAdapt performs iterative hierarchical
classification, expanding both the taxonomy width and depth based on corpus'
topical distribution. We demonstrate its state-of-the-art performance across a
diverse set of computer science conferences over the years to showcase its
ability to structure and capture the evolution of scientific fields. As a
multidimensional method, TaxoAdapt generates taxonomies that are 26.51% more
granularity-preserving and 50.41% more coherent than the most competitive
baselines judged by LLMs.
Claims made by individuals or entities are oftentimes nuanced and cannot be
clearly labeled as entirely "true" or "false" -- as is frequently the case with
scientific and political claims. However, a claim (e.g., "vaccine A is better
than vaccine B") can be dissected into its integral aspects and sub-aspects
(e.g., efficacy, safety, distribution), which are individually easier to
validate. This enables a more comprehensive, structured response that provides
a well-rounded perspective on a given problem while also allowing the reader to
prioritize specific angles of interest within the claim (e.g., safety towards
children). Thus, we propose ClaimSpect, a retrieval-augmented generation-based
framework for automatically constructing a hierarchy of aspects typically
considered when addressing a claim and enriching them with corpus-specific
perspectives. This structure hierarchically partitions an input corpus to
retrieve relevant segments, which assist in discovering new sub-aspects.
Moreover, these segments enable the discovery of varying perspectives towards
an aspect of the claim (e.g., support, neutral, or oppose) and their respective
prevalence (e.g., "how many biomedical papers believe vaccine A is more
transportable than B?"). We apply ClaimSpect to a wide variety of real-world
scientific and political claims featured in our constructed dataset, showcasing
its robustness and accuracy in deconstructing a nuanced claim and representing
perspectives within a corpus. Through real-world case studies and human
evaluation, we validate its effectiveness over multiple baselines.
Wang Xinjie, Liu Liu, Cao Yu, Wu Ruiqi, Qin Wenkang, Wang Dehui, Sui Wei, Su Zhizhong
22
Constructing a physically realistic and accurately scaled simulated 3D world
is crucial for the training and evaluation of embodied intelligence tasks. The
diversity, realism, low cost accessibility and affordability of 3D data assets
are critical for achieving generalization and scalability in embodied AI.
However, most current embodied intelligence tasks still rely heavily on
traditional 3D computer graphics assets manually created and annotated, which
suffer from high production costs and limited realism. These limitations
significantly hinder the scalability of data driven approaches. We present
EmbodiedGen, a foundational platform for interactive 3D world generation. It
enables the scalable generation of high-quality, controllable and
photorealistic 3D assets with accurate physical properties and real-world scale
in the Unified Robotics Description Format (URDF) at low cost. These assets can
be directly imported into various physics simulation engines for fine-grained
physical control, supporting downstream tasks in training and evaluation.
EmbodiedGen is an easy-to-use, full-featured toolkit composed of six key
modules: Image-to-3D, Text-to-3D, Texture Generation, Articulated Object
Generation, Scene Generation and Layout Generation. EmbodiedGen generates
diverse and interactive 3D worlds composed of generative 3D assets, leveraging
generative AI to address the challenges of generalization and evaluation to the
needs of embodied intelligence related research. Code is available at
https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html.
Faithful evaluation of language model capabilities is crucial for deriving
actionable insights that can inform model development. However, rigorous causal
evaluations in this domain face significant methodological challenges,
including complex confounding effects and prohibitive computational costs
associated with extensive retraining. To tackle these challenges, we propose a
causal representation learning framework wherein observed benchmark performance
is modeled as a linear transformation of a few latent capability factors.
Crucially, these latent factors are identified as causally interrelated after
appropriately controlling for the base model as a common confounder. Applying
this approach to a comprehensive dataset encompassing over 1500 models
evaluated across six benchmarks from the Open LLM Leaderboard, we identify a
concise three-node linear causal structure that reliably explains the observed
performance variations. Further interpretation of this causal structure
provides substantial scientific insights beyond simple numerical rankings:
specifically, we reveal a clear causal direction starting from general
problem-solving capabilities, advancing through instruction-following
proficiency, and culminating in mathematical reasoning ability. Our results
underscore the essential role of carefully controlling base model variations
during evaluation, a step critical to accurately uncovering the underlying
causal relationships among latent model capabilities.
Yuan Yuan, Yukun Liu, Chonghua Han, Jie Feng, Yong Li
22
Foundation models have revolutionized fields such as natural language
processing and computer vision by enabling general-purpose learning across
diverse tasks and datasets. However, building analogous models for human
mobility remains challenging due to the privacy-sensitive nature of mobility
data and the resulting data silos across institutions. To bridge this gap, we
propose MoveGCL, a scalable and privacy-preserving framework for training
mobility foundation models via generative continual learning. Without sharing
raw data, MoveGCL enables decentralized and progressive model evolution by
replaying synthetic trajectories generated from a frozen teacher model, and
reinforces knowledge retention through a tailored distillation strategy that
mitigates catastrophic forgetting. To address the heterogeneity of mobility
patterns, MoveGCL incorporates a Mixture-of-Experts Transformer with a
mobility-aware expert routing mechanism, and employs a layer-wise progressive
adaptation strategy to stabilize continual updates. Experiments on six
real-world urban datasets demonstrate that MoveGCL achieves performance
comparable to joint training and significantly outperforms federated learning
baselines, while offering strong privacy protection. MoveGCL marks a crucial
step toward unlocking foundation models for mobility, offering a practical
blueprint for open, scalable, and privacy-preserving model development in the
era of foundation models.
Ho Yin 'Sam' Ng, Ting-Yao Hsu, Aashish Anantha Ramakrishnan, Branislav Kveton, Nedim Lipka, Franck Dernoncourt, Dongwon Lee, Tong Yu, Sungchul Kim, Ryan A. Rossi, Ting-Hao 'Kenneth' Huang
12
Figure captions are crucial for helping readers understand and remember a
figure's key message. Many models have been developed to generate these
captions, helping authors compose better quality captions more easily. Yet,
authors almost always need to revise generic AI-generated captions to match
their writing style and the domain's style, highlighting the need for
personalization. Despite language models' personalization (LaMP) advances,
these technologies often focus on text-only settings and rarely address
scenarios where both inputs and profiles are multimodal. This paper introduces
LaMP-Cap, a dataset for personalized figure caption generation with multimodal
figure profiles. For each target figure, LaMP-Cap provides not only the needed
inputs, such as figure images, but also up to three other figures from the same
document--each with its image, caption, and figure-mentioning paragraphs--as a
profile to characterize the context. Experiments with four LLMs show that using
profile information consistently helps generate captions closer to the original
author-written ones. Ablation studies reveal that images in the profile are
more helpful than figure-mentioning paragraphs, highlighting the advantage of
using multimodal profiles over text-only ones.
As automated attack techniques rapidly advance, CAPTCHAs remain a critical
defense mechanism against malicious bots. However, existing CAPTCHA schemes
encompass a diverse range of modalities -- from static distorted text and
obfuscated images to interactive clicks, sliding puzzles, and logic-based
questions -- yet the community still lacks a unified, large-scale, multimodal
benchmark to rigorously evaluate their security robustness. To address this
gap, we introduce MCA-Bench, a comprehensive and reproducible benchmarking
suite that integrates heterogeneous CAPTCHA types into a single evaluation
protocol. Leveraging a shared vision-language model backbone, we fine-tune
specialized cracking agents for each CAPTCHA category, enabling consistent,
cross-modal assessments. Extensive experiments reveal that MCA-Bench
effectively maps the vulnerability spectrum of modern CAPTCHA designs under
varied attack settings, and crucially offers the first quantitative analysis of
how challenge complexity, interaction depth, and model solvability interrelate.
Based on these findings, we propose three actionable design principles and
identify key open challenges, laying the groundwork for systematic CAPTCHA
hardening, fair benchmarking, and broader community collaboration. Datasets and
code are available online.
Real-time reconstruction of dynamic 3D scenes from uncalibrated video streams
is crucial for numerous real-world applications. However, existing methods
struggle to jointly address three key challenges: 1) processing uncalibrated
inputs in real time, 2) accurately modeling dynamic scene evolution, and 3)
maintaining long-term stability and computational efficiency. To this end, we
introduce StreamSplat, the first fully feed-forward framework that transforms
uncalibrated video streams of arbitrary length into dynamic 3D Gaussian
Splatting (3DGS) representations in an online manner, capable of recovering
scene dynamics from temporally local observations. We propose two key technical
innovations: a probabilistic sampling mechanism in the static encoder for 3DGS
position prediction, and a bidirectional deformation field in the dynamic
decoder that enables robust and efficient dynamic modeling. Extensive
experiments on static and dynamic benchmarks demonstrate that StreamSplat
consistently outperforms prior works in both reconstruction quality and dynamic
scene modeling, while uniquely supporting online reconstruction of arbitrarily
long video streams. Code and models are available at
https://github.com/nickwzk/StreamSplat.