Research papers

Bin Chen, Shouliang Zhu, Beidan Liu, Yong Zhao +3 more

Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal hum…

Yaxin Du, Junru Song, Yifan Zhou, Cheng Wang +7 more

Retrieval-augmented generation is a practical paradigm for question answering over long documents, but it remains brittle for multimodal reading where text, tables, and figures are interleaved across many pages. First, flat chunking breaks document-native structure and cross-modal alignment, yielding semantic fragments…

Albert Tseng, Chris De Sa

Modern sparse language models typically achieve sparsity through Mixture-of-Experts (MoE) layers, which dynamically route tokens to dense MLP "experts." However, dynamic hard routing has a number of drawbacks, such as potentially poor hardware efficiency and needing auxiliary losses for stable training. In contrast, t…

Quanxin Liu, Rui Hao, Ruida Xu, Jianwei Zhong +2 more

End-to-end data science agent workflows involve tightly coupled sub-processes with strong dynamic dependencies, posing a challenging long-horizon orchestration problem. Existing frameworks primarily rely on static, chain-like execution plans, which are prone to error propagation from early stages—often causing reasonin…

Yi-Kai Zhang, Zhiyuan Yao, Hongyan Hao, Yueqing Sun +5 more

Policy gradient methods rely on a baseline to measure the relative advantage of an action. In Actor-Critic LLM training (e.g., PPO), this baseline is typically estimated by a Value Model as large as the policy. However, as the policy evolves, the value model requires expensive, synchronous training. To avoid t…

Herilalaina Rakotoarison, Steven Adriaensen, Tom Viering, Carl Hvarfner +3 more

Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration–exploitation framework for Bayesian optimization (BO). However, their practical implementation relies on complicated and slow approximations, i.e., a Monte Carlo estimation of the information gain. This complexity can…

Zihao Wang, Hang Yin, Lihui Liu, Hanghang Tong +3 more

This paper studies the Minimal Embeddable Dimension (MED): the least dimension in which there exists a configuration of $m$ object vectors so that every subset of size at most $k$ is exactly retrieved by score comparison. Our result shows MED is $\Theta(k)$, independent of $m$, for inner product, Euclidean distance, an…

Min ZHOU, Enming Liang, Minghua Chen

Learning permutation-invariant functions over sets of $N$ elements, where the output is independent of the input ordering, is fundamental to many deep learning applications. While sum-decomposable architectures like DeepSets offer universal approximation for such functions, existing constructive bounds require a latent…

Yuxin Ma, Nan Chen, Mateo D Diaz, Soufiane Hayou +2 more

Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored *model upscaling*: initializing larger models from trained smaller ones to accelerate convergence. However, this method can be sensitive to hyperp…

Lizhang Chen, Jonathan Li, Qi Wang, Runlong Liao +4 more

Mixture-of-Experts (MoE) models rely on balanced expert utilization to fully realize their scalability. However, existing load-balancing methods are largely heuristic and operate on noisy mini-batch assignment statistics, introducing bias relative to population-level objectives. We propose **$\phi$-balancing**, a princ…

Bingxuan Zhao, Qing Zhou, Yu Wang, Chuang Yang +1 more

Diffusion Transformers (DiTs) can synthesize high-fidelity images, but training at ultra-high resolutions is expensive, making inference-time extrapolation essential. Existing methods are typically scale-agnostic, applying the same positional-modulation schedule regardless of target resolution. We show that th…

Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan +1 more

Conversational agents are increasingly deployed in knowledge-intensive settings, where correct behavior depends on retrieving and applying domain-specific knowledge from large, proprietary, and unstructured corpora during live interactions with users. Yet most existing benchmarks evaluate retrieval or tool use independ…

Soham Ray, Keshav Dhandhania, Victor Barres, Karthik Narasimhan

Full-duplex voice agents—systems that listen and speak simultaneously—are rapidly moving from research to production. However, existing evaluations address conversational dynamics and task completion in isolation. We introduce τ-Voice, a benchmark for evaluating voice agents on grounded tasks with real-world complexity…

Victor Barres, Honghua Dong, Soham Ray, Xujie Si +1 more

Existing benchmarks for conversational AI agents simulate *single-control* environments, where only the AI agent can use tools to interact with the world, while the user remains a passive information provider. This differs from real-world scenarios like technical support, where users need to actively participate in mod…

Xiwen Chen, Wenhui Zhu, Jingjing Wang, Peijie Qiu +12 more

Aligning Large Language Models (LLMs) with human preferences is often formulated via Direct Preference Optimization (DPO). However, the standard Bradley-Terry instantiation of DPO is limited in modeling common departures from transitivity in human preferences. To address this, recent work has introduced Self-Play Prefe…

Mayank Kumar, Qian Lou, Paulo Barreto, Martine De Cock +1 more

Data is the lifeblood of AI, yet much of the most valuable data remains locked in silos due to privacy and regulations. As a result, AI remains heavily underutilized in many of the most important domains, including healthcare, education, and finance. Synthetic data generation (SDG), i.e. the generation of artificial da…

Pingzhi Li, Hongxuan Li, Zirui Liu, Xingcheng Lin +1 more

Graph neural network (GNN) potentials such as SchNet improve the accuracy and transferability of molecular dynamics (MD) simulation by learning many-body interactions, but remain slower than classical force fields due to fragmented kernels and memory-bound pipelines that underutilize GPUs. We show that a missing pr…

Muqi Han, Ruoqi Xing, KAI WU, Xiaoyu Zhang +2 more

Meta Black-Box Optimization (MetaBBO) has emerged as a promising paradigm by employing meta learning to automatically optimize the configurations of low-level black-box optimizers. Despite its potential, the generalization of MetaBBO remains significantly constrained when facing unseen, complex objective landscapes. We…

Sangeun Park, Minhae Kwon

A central goal of large language model (LLM) research is to build agentic systems that can plan, act, and adapt through sustained interaction with dynamic environments. While recent LLM-based agents exhibit impressive contextual reasoning, their long-horizon decision-making remains fragile, often suffering from $\texti…

Yining Jiao, Shankar Bhamidi, Carlton ZDANSKI, Julia Kimbell +8 more

Understanding how anatomical shapes evolve in response to developmental covariates—and quantifying their spatially varying uncertainties—is critical in healthcare research. Existing approaches typically rely on global time-warping formulations that ignore spatially heterogeneous dynamics. We introduce PRISM, a…