Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the datasets used. We investigate themes of dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups: dataset design, dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.
@article{Mallahnezhad2026TowardsDC,author={Mahdieh Mallahnezhad and
Masoud Jafaripour and
Mohamadali Tofigh and
Sheila Schoepp and
Osmar R. Zaiane},title={Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models},journal={arXiv},year={2026},month=oct,day={13},url={https://arxiv.org/abs/2610.19108},doi={10.48550/arXiv.2610.19108},eprinttype={arXiv},eprint={2610.19108}}
Preprint
AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot
Scientific peer review faces mounting strain as submission volumes surge, making it increasingly difficult to sustain review quality, consistency, and timeliness. Recent advances in AI have led the community to consider its use in peer review, yet a key unresolved question is whether AI can generate technically sound reviews at real-world conference scale. Here we report the first large-scale field deployment of AI-assisted peer review: every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system. The system combined frontier models, tool use, and safeguards in a multi-stage process to generate reviews for all 22,977 full-review papers in less than a day. A large-scale survey of AAAI-26 authors and program committee members showed that participants not only found AI reviews useful, but actually preferred them to human reviews on key dimensions such as technical accuracy and research suggestions. We also introduce a novel benchmark and find that our system substantially outperforms a simple LLM-generated review baseline at detecting a variety of scientific weaknesses. Together, these results show that state-of-the-art AI methods can already make meaningful contributions to scientific peer review at conference scale, opening a path toward the next generation of synergistic human-AI teaming for evaluating research.
@article{Biswas2026AIAssistedPR,author={Joydeep Biswas and
Sheila Schoepp and
Gautham Vasan and
Anthony Opipari and
Arthur Zhang and
Zichao Hu and
Sebastian Joseph and
Matthew Lease and
Junyi Jessy Li and
Peter Stone and
Kiri L. Wagstaff and
Matthew E. Taylor and
Odest Chadwicke Jenkins},title={AI-Assisted Peer Review at Scale: The {AAAI-26} {AI} Review Pilot},journal={CoRR},volume={abs/2604.13940},year={2026},url={https://doi.org/10.48550/arXiv.2604.13940},doi={10.48550/ARXIV.2604.13940},eprinttype={arXiv},eprint={2604.13940},timestamp={Mon, 11 May 2026 17:08:47 +0200},biburl={https://dblp.org/rec/journals/corr/abs-2604-13940.bib},bibsource={dblp computer science bibliography, https://dblp.org},month=apr}
2025
IJCAI
The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning
Reinforcement learning (RL) has shown impressive results in sequential decision-making tasks. Meanwhile, Large Language Models (LLMs) and Vision-Language Models (VLMs) have emerged, exhibiting impressive capabilities in multimodal understanding and reasoning. These advances have led to a surge of research integrating LLMs and VLMs into RL. In this survey, we review representative works in which LLMs and VLMs are used to overcome key challenges in RL, such as lack of prior knowledge, long-horizon planning, and reward design. We present a taxonomy that categorizes these LLM/VLM-assisted RL approaches into three roles: agent, planner, and reward. We conclude by exploring open problems, including grounding, bias mitigation, improved representations, and action advice. By consolidating existing research and identifying future directions, this survey establishes a framework for integrating LLMs and VLMs into RL, advancing approaches that unify natural language and visual understanding with sequential decision-making.
@inproceedings{Schoepp2025TheEL,author={Sheila Schoepp and
Masoud Jafaripour and
Yingyue Cao and
Tianpei Yang and
Fatemeh Abdollahi and
Shadan Golestan and
Zahin Sufiyan and
Osmar R. Za{\"{\i}}ane and
Matthew E. Taylor},title={The Evolving Landscape of {LLM-} and VLM-Integrated Reinforcement Learning},booktitle={Proceedings of the International Joint Conference on Artificial Intelligence ({IJCAI})},pages={10641--10649},publisher={ijcai.org},year={2025},url={https://doi.org/10.24963/ijcai.2025/1181},doi={10.24963/IJCAI.2025/1181},timestamp={Wed, 24 Sep 2025 17:45:28 +0200},biburl={https://dblp.org/rec/conf/ijcai/SchoeppJCYAGSZT25.bib},bibsource={dblp computer science bibliography, https://dblp.org}}
2024
Preprint
Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms
Industry is rapidly moving towards fully autonomous and interconnected systems that can detect and adapt to changing conditions, including machine hardware faults. Traditional methods for adding hardware fault tolerance to machines involve duplicating components and algorithmically reconfiguring a machine’s processes when a fault occurs. However, the growing interest in reinforcement learning-based robotic control offers a new perspective on achieving hardware fault tolerance. However, limited research has explored the potential of these approaches for hardware fault tolerance in machines. This paper investigates the potential of two state-of-the-art reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC), to enhance hardware fault tolerance into machines. We assess the performance of these algorithms in two OpenAI Gym simulated environments, Ant-v2 and FetchReach-v1. Robot models in these environments are subjected to six simulated hardware faults. Additionally, we conduct an ablation study to determine the optimal method for transferring an agent’s knowledge, acquired through learning in a normal (pre-fault) environment, to a (post-)fault environment in a continual learning setting. Our results demonstrate that reinforcement learning-based approaches can enhance hardware fault tolerance in simulated machines, with adaptation occurring within minutes. Specifically, PPO exhibits the fastest adaptation when retaining the knowledge within its models, while SAC performs best when discarding all acquired knowledge. Overall, this study highlights the potential of reinforcement learning-based approaches, such as PPO and SAC, for hardware fault tolerance in machines. These findings pave the way for the development of robust and adaptive machines capable of effectively operating in real-world scenarios.
@article{Schoepp2024EnhancingHF,author={Sheila Schoepp and
Mehran Taghian and
Shotaro Miwa and
Yoshihiro Mitsuka and
Shadan Golestan and
Osmar Za{\"{\i}}ane},title={Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms},journal={CoRR},volume={abs/2407.15283},year={2024},url={https://doi.org/10.48550/arXiv.2407.15283},doi={10.48550/ARXIV.2407.15283},eprinttype={arXiv},eprint={2407.15283},timestamp={Mon, 19 Aug 2024 21:11:14 +0200},biburl={https://dblp.org/rec/journals/corr/abs-2407-15283.bib},bibsource={dblp computer science bibliography, https://dblp.org},month=jul}