Faculty
Dr. Stefano V. Albrecht
Associate Professor
Head of Research Group
Postdoctoral Researchers
Dr. Xue Yan
PhD in Pattern Recognition and Intelligent Systems, Institute of Automation, Chinese Academy of Sciences, 2026; BEng in Computer Science and Technology, Shandong University, 2021
Project: Beyond Token-Level Optimization: High-Level Credit Assignment for LLM Reinforcement Learning
Research interests: RL, LLM-based agents, human-AI coordination, multi-agent systems
Dr. Zhao Feng
PhD in Biomedical Engineering, Zhejiang University, 2025; BS in Biomedical Engineering, Zhejiang University, 2020
Project: Modelling Cognitive Decline and Recovery in Multiple Sclerosis Using LLMs and Reinforcement Learning
Research interests: neuroengineering, brain-computer interface
PhD Research Students
Eason Yu
MSc Computer Science, University of Sydney, 2026; BA Information Management, National Taiwan University, 2023
Project: Auto-Curriculum Learning via Self-Play
In reinforcement learning, a key challenge is that an agent's intelligence ceiling is often bounded by the environment. In practice, simple tasks are easy to learn but tend to produce uninteresting behaviors, while more complex tasks may remain entirely out of reach. Self-play provides an elegant solution where the complexity of the environment is defined by the agent itself and co-evolves over time. My research aims to develop efficient self-play algorithms, together with suitable model architectures, to enable the emergence of intelligent agents.
Ziyuan Liu
Master of Information Systems Management, Carnegie Mellon University, 2022; BSc Information Systems, Singapore Management University, 2021
Project: Inter-Agent Alignment in Ad Hoc Multi-Agent Workflows
In ad hoc teamwork, agents must collaborate with previously unseen teammates without prior coordination or complete knowledge of their capabilities and requirements. In complex workflows, successful collaboration can hinge on achieving alignment among agents regarding each other's capabilities, responsibilities and needs. In this project, we investigate how agents can establish and maintain alignment, surface hidden constraints, and adapt the structure of the workflow to improve coordination with unfamiliar teammates.
Tianyi Tang
MSc in Artificial Intelligence, Nanyang Technological University, 2025; BMath (Hons) in Computational Mathematics, University of Waterloo, 2023
Project: Governable Organizational Policies for Multi-Agent Workflow Orchestration
A key challenge in multi-agent systems is enabling Manager Agents to coordinate heterogeneous workers while satisfying safety, compliance, resource, and stakeholder requirements. My project investigates how explicit organizational policies can guide Manager Agents in decomposing complex tasks, coordinating workers, managing dependencies, and responding to failures throughout long-horizon workflows. The goal is to develop reliable and adaptable Manager Agents that can maintain effective collaboration and constraint compliance in dynamic, partially observable environments.
Tanisha Annamalai
BSc (Hons) Biological Sciences, Nanyang Technological University, 2024
Project: Modelling Multiple Sclerosis Cognitive Decline Using LLMs and Reinforcement Learning
Multiple sclerosis (MS) is a chronic inflammatory and demyelinating disease of the central nervous system causing progressive cognitive and motor deficits. Despite its prevalence, the mechanisms underlying MS cognitive decline and recovery remain poorly understood. This project proposes a novel computational framework employing a large language model (LLM) as a proxy for the human brain to simulate MS-related neurodegeneration. A reinforcement learning-based recovery process will be explored to identify optimal windows for early intervention to mitigate cognitive decline.
Yize Liu
BSc (Hons) Quantitative Finance, National University of Singapore, 2026
Project: LLM Agents in Multi-Agent Decision-Making
Large language model agents are increasingly being used in multi-agent systems, where they must communicate, coordinate, adapt, and make strategic decisions with other agents. However, their performance is often inconsistent across different tasks, interaction structures, and incentive settings. These limitations make it difficult to determine whether LLM agents possess robust multi-agent capabilities or merely exploit task-specific patterns. This project will systematically evaluate LLM agents across cooperative, competitive, and mixed-motive environments, identify their key failure modes, and develop principled empirical and game-theoretic methods for improving their coordination, generalisation, and decision-making.
Ka Yiu Lee
MSc Data Science and Machine Learning, University College London, 2023; BEng Engineering Mathematics, University of Bristol, 2019
Project: General-Purpose Agents that Continue Learning at Test Time
Learning systems are largely static once trained: adapting them to a new task usually requires retraining rather than experience. I am interested in general-purpose agents that keep acquiring skills after training has ended. My focus is continual learning and test-time learning: how an agent can integrate new experience without overwriting what it already knows, and adapt to tasks it has never encountered by spending extra computation, interaction, or feedback at inference time. My work draws on reinforcement learning and open-ended learning, with the longer-term goal of agents that exceed human capability across a broad range of tasks.
Jiawen Zong
Bachelor of Engineering in Information Engineering and Media, Nanyang Technological University, 2026
Project: Theory of Mind Multi-Agent Reinforcement Learning for Safe Collaborative Robotics
Multi-agent robotic teams often struggle to maintain safety when external collaborators unpredictably change their behaviors midtask. While current intent detection models can track these behavioral shifts, their passive nature fails to dynamically coordinate the team's joint actions. This project introduces a framework that integrates real-time belief state tracking of external intents directly into a decentralized multi-agent reinforcement learning control loop, improving both collaborative adaptability and collision avoidance.
Visiting Researchers
Aadhithyan Velan M.
BE Computer Science, BITS Pilani, Hyderabad Campus, 2027
Project: A Unified Fairness Framework for Sequential Social Dilemmas in Multi-Agent Reinforcement Learning
A sequential social dilemma (SSD) is a scenario where agents interacting over time face a tension between actions that benefit themselves and actions that benefit the collective good. Fairness in an SSD needs mechanisms that go beyond linear objectives like the maximization of collective reward, and an asymmetric SSD (where agents differ structurally) needs this even more, since equal treatment can still produce unequal outcomes. Our work aims to create a framework that identifies what local, decentralized information (neighbor signals, reward-sharing, public broadcasts) is necessary and sufficient for agents to approximate global fairness objectives without centralized oversight.
Ian Bryant
BS Computer Science & BS Applied and Computational Mathematics, University of California, Irvine
Project: Scaling Pareto-Optimal Equilibrium Selection in Multi-Agent Reinforcement Learning
Multi-Agent Reinforcement Learning (MARL) algorithms can avoid poor equilibria when they train agents to choose Pareto-optimal actions. However, naive approaches in order to do so typically require evaluating the set of all possible joint actions. The size of this set is exponential in the number of agents and the dimensionality of each agent's action space, causing methods to scale poorly. This project explores alternative techniques to group and evaluate similar joint actions together, rather than individually, improving the scalability of calculating the Pareto-optimal actions.
Yucong Huang
Master of Engineering Management, Peking University, 2025
Project: Self-Evolving Agents for Cross-Device Kernel Generation and Optimisation
Generating high-performance kernels across heterogeneous GPUs and accelerators remains challenging because optimisation experience is often tightly coupled with specific hardware architectures, programming models, kernel languages, and performance characteristics. My project investigates self-evolving agents that can extract transferable knowledge from previous kernel-generation and optimisation trajectories, adapt it to new devices, and refine it through compilation, execution, and profiling feedback. The longer-term goal is to develop general mechanisms that enable agents to share and reuse heterogeneous experience across devices, tasks, and application domains.
University of Edinburgh Alumni
Before moving to NTU, the group was based in the School of Informatics at the University of Edinburgh. Below is a list of people who were affiliated with the group while based at Edinburgh.
Honorary Fellow (Edinburgh)
Dr. David Abel
Honorary Fellow, University of Edinburgh
Senior Research Scientist at Google DeepMind
Postdoctoral Researchers (Edinburgh)
Dr. Alper Demir (Aug 2024 – July 2025)
PhD in Computer Engineering, 2019; MSc in Computer Engineering, 2016; BSc in Computer Engineering, 2014, Middle East Technical University
Project: Solving Asymmetric Social Dilemmas in Multi-Agent Reinforcement Learning
Research interests: reinforcement learning, multi-agent systems, data science
Dr. Atish Dixit (Nov 2023 – Nov 2024)
PhD in Engineering, Heriot-Watt University, 2023; MTech in Mathematical Modelling, 2017 & BEng in Mechanical Engineering, 2012, University of Pune
Project: Multi-Agent Reinforcement Learning and Ad Hoc Multi-Agent Collaboration
Research interests: reinforcement learning, multi-agent systems, robotics, intelligent control
Dr. Dongge Han (Aug 2023 – May 2024)
PhD in Computer Science, University of Oxford, 2022; MSc in Computer Science, University of Oxford, 2016; BSc in Physics, The Hong Kong University of Science and Technology, 2015
Project: Multimodal Integration for Sample-Efficient Deep Reinforcement Learning
Research interests: reinforcement learning, recommender systems, multi-agent systems
Dr. Cheng Wang (Oct 2022 – April 2024)
PhD in Engineering, Technical University of Darmstadt, 2021; MS in Automotive Engineering, Tongji University, 2017; BS in Automotive Engineering, Wuhan University of Technology, 2014
Project: Asking Your Autonomous Car to Explain its Decisions: Towards AI That Explains Itself
Research interests: prediction and planning, simulation and testing, safety verification and validation, autonomous vehicles
Dr. Ignacio Carlucho (Nov 2021 – April 2023)
PhD in Engineering, National University of Central Buenos Aires, 2019; BS in Electromechanical Engineering, National University of Central Buenos Aires, 2015
Project: Explainable Reasoning, Learning and Ad hoc Multi-agent Collaboration
Research interests: reinforcement learning, multi-agent systems, robotics, intelligent control
Dr. Josiah P. Hanna (Jan 2019 – July 2020)
PhD in Computer Science, University of Texas at Austin, 2019; BS in Computer Science and Mathematics, University of Kentucky, 2014
Project: Towards Model Criticism in Multi-Agent Systems
Research interests: reinforcement learning, policy evaluation, robotics, autonomous driving
PhD Research Students (Edinburgh)
Leonard Hinckeldey (Nov 2023 – supervised until Jan 2025)
MSc Applied Social Data Science, London School of Economics, 2023; BSc Economics, SOAS University of London, 2022
Project: Collaborative Multi-Agent Reinforcement Learning for Ad-Hoc Human-AI Teams
Multi-Agent Reinforcement Learning (MARL) holds great potential for coordinating the behaviour of artificial agents in complex real-world environments. However, MARL agents often perform poorly when partnered with agents previously unseen during training, which hampers their deployment in real-world environments that involve human agents. This project explores how MARL algorithms can learn generalisable policies, enabling effective and spontaneous collaboration between RL agents and humans.
Kale-ab Tessera (Sep 2023 – supervised until Jan 2025)
MSc Computer Science, University of the Witwatersrand, 2021; BSc Hons in Computer Science, University of Pretoria, 2016
Project: Scalable Coordination in Multi-Agent Reinforcement Learning
In Multi-Agent Reinforcement Learning (MARL), scalable coordination has historically been a significant challenge, limiting the real-world applicability of these algorithms. With the increasing deployment of machine learning systems, the need for scalable MARL methods that can interact with diverse agents becomes even more crucial. Our research focuses on improving the scalability of MARL algorithms, while also ensuring these algorithms can adapt to seen and unseen agents.
Raul D. Steleac (Sep 2023 – supervised until Jan 2025)
MSc in Computing (Artificial Intelligence and Machine Learning), Imperial College London, 2021
Project: Continual Multi-Agent Reinforcement Learning
Addressing the challenge of adaptability in multi-agent reinforcement learning, this project aims to improve agents' abilities to cooperate with diverse and changing teammates after deployment. By exposing the learner to various team configurations during training, current approaches assume that sufficient adaptability is instilled in the agent to develop a general enough cooperation capacity; however, these methods often fail to cover the entire teammate policy space in complex scenarios. This project introduces Continual Multi-Agent Reinforcement Learning to extend learning capabilities of agents beyond the initial training period, allowing them to effectively adjust to previously unseen team configurations and strategies.
Elle Miller (Sep 2023 – supervised until Jan 2025)
Bachelor of Mechatronic (Space) Engineering & Bachelor of Advanced Science (Physics), University of Sydney, 2023
Project: Deep Reinforcement Learning for Safe and Compliant Human-Robot Interaction
Robots possess significant potential to enhance the quality of life for individuals with disabilities, support healthcare professionals, and provide care to an ageing population. To help with tasks such as drinking, dressing, and personal hygiene, robots will need to perform very intricate behaviours while in direct physical contact with humans. Deep Reinforcement Learning (DRL) has emerged as a promising avenue to acquire diverse complex behaviours safely through simulation. However, an open challenge lies in transferring these learned policies to real-world scenarios while ensuring safety. In this project, we investigate the potential of DRL to learn safe and compliant assistive behaviours for physical human-robot interaction.
Trevor McInroe (Oct 2022 – supervised until Jan 2025)
MS Artificial Intelligence, Northwestern University, 2022; BBA Economics, University of North Texas, 2017
Project: Enabling Real-World Offline Reinforcement Learning with Representation Learning
The sample complexity of reinforcement learning (RL) algorithms hinders their application to real-world systems. This inefficiency is exacerbated by high-dimensional state spaces, such as those composed of pixels. My research will investigate how representation learning routines can disentangle useful information from small offline datasets of robotic tasks. My ultimate goal is to make RL feasible for the average industry group in the same way that computer vision has become accessible over the past decade.
Sabrina McCallum (Sep 2022 – supervised until Jan 2025)
MSc Artificial Intelligence, University of Strathclyde, 2022; BA Business Administration, Berlin School of Economics and Law, 2015
Project: Learning Grounded Representations from Multi-Modal Feedback and Interactions with Embodied Environments
Learning complex, hierarchical tasks or diverse, open-ended tasks when rewards are sparse or there is no clear success criterion remains a challenge for RL agents. Manually crafting dense shaping rewards is non-trivial and even potentially infeasible for some environments, and choosing a good heuristic requires domain knowledge, typically resulting in task- and environment-specific solutions. This project explores alternative approaches which instead leverage information-rich language feedback and other multi-modal signals resulting directly from interactions of embodied agents with their environment.
Samuel Garcin (Sep 2021 – supervised until Jan 2025)
MEng in Aeronautical Engineering, Imperial College London, 2018
Project: Adaptive Curriculum Design for Generalisation in Deep Reinforcement Learning
A key limitation preventing the wider adoption of Deep Reinforcement Learning (DRL) today is its difficulty generalising to environments or tasks which were not encountered during training. The frameworks put forward to tackle this issue, such as Meta Reinforcement Learning or Representation Learning, primarily focus on the DRL agent and do not act on the training task generation process. This project will investigate how Representation Learning methods may be employed to learn a structured latent representation of the problem class to be solved, enabling the generation of an adaptive task distribution that captures the problem class, and is adapted to the current level of ability of the agent.
Balint Gyevnar (Sep 2021 – Jan 2026)
MInf Informatics, University of Edinburgh, 2021
Project: Natural Language Explanations for Autonomous Vehicle Motion Planning and Prediction
Achieving trust and safety for autonomous vehicles is critical to their public success. However, most current methods rely on opaque and unaccountable black-box algorithms, making their legal and social adoption difficult. Instead, grounded in interpretable and explainable methods such as IGP2 and integrated with natural language processing and cognitive modelling, my project will investigate how to generate and deliver the most relevant and intelligible explanations for users, with the end goal of building trust and transparency in autonomous vehicles.
Mhairi Dunion (Sep 2020 – Feb 2025)
BSc (Hons) Mathematics, University of Edinburgh, 2013
Project: Causality in Deep Reinforcement Learning
A challenge of deep reinforcement learning is that it does not generalise to unseen tasks with the same underlying dynamics because it overfits to the training task. In practice, it is common to train algorithms with random initialisations of all environment variables to maximise the tasks seen during training, which is not pragmatic or sample efficient. This project will investigate novel methods to improve generalisation to unseen tasks by combining causal inference techniques with deep reinforcement learning because causal relationships remain invariant to the change in task.
Elliot Fosong (Sep 2019 – March 2025)
BA & MEng in Engineering, University of Cambridge, 2019
Project: Coordination of Pre-skilled Agents to Complete Unseen Tasks
Teams of autonomous agents can be trained to complete specific desirable tasks. When a new task arises, we might wish to form a new team to complete this task by selecting existing agents whose skills could be useful to solve the new task. Despite the agents' individual skills in respective roles, the newly formed team needs to learn to coordinate to solve the new task. This project aims to develop methods by which the already-skilled agents can learn to cooperate to solve a new task, given a limited number of 'trial runs' on the new task.
Lukas Schäfer (Dec 2019 – Oct 2024)
MSc Informatics, University of Edinburgh, 2019; BSc Computer Science, Saarland University, 2018
Project: Sample Efficiency and Generalisation in Multi-Agent Reinforcement Learning
Multi-agent reinforcement learning systems often require many millions of interactions to learn complex behaviour. Furthermore, the learned behaviour usually lacks generalisation ability. These challenges of sample efficiency and generalisation together severely limit the possible application of multi-agent reinforcement learning. This project will leverage distributed information and the multi-agent nature of such systems to enable agents to learn effective behaviours with less data and be able to learn robust, re-usable skills which transfer to new environments.
Cillian Brewitt (Jan 2019 – May 2023)
MSc Artificial Intelligence, University of Edinburgh, 2017; BE Electrical and Electronic Engineering, University College Cork, 2016
Project: Interpretable Planning and Prediction for Autonomous Vehicles
Accurately predicting the intentions and actions of other road users and then using this information during motion planning is an important task in the field of autonomous driving. It is desirable for planning and prediction methods to be fast, accurate, interpretable, and verifiable, however current methods fail to achieve all these objectives. During this project novel methods for prediction and planning which satisfy these objectives will be investigated. My current focus is investigating how decision trees can be used for vehicle goal recognition.
Filippos Christianos (Sep 2018 – Jun 2023)
Diploma in Electronic and Computer Engineering, Technical University of Crete, 2017
Project: Coordinated Exploration in Multi-Agent Deep Reinforcement Learning
In the increasingly large state space encountered in deep reinforcement learning, exploration plays a critical role by narrowing down the search for an optimal policy. In multi-agent settings, the joint action space also grows exponentially, further complicating the search. The use of a partially centralized policy while exploring can coordinate the exploration and more easily locate promising, even decentralized, policies. In this project, we investigate how the coordination of agents in the exploration phase can improve the performance of deep reinforcement learning algorithms.
Georgios Papoudakis (Sep 2018 – Mar 2024)
Diploma in Electrical and Computer Engineering, Aristotle University of Thessaloniki, 2017
Project: Modelling in Multi-Agent Systems Using Representation Learning
Multi-agent systems in partially observable environments face many challenging problems which traditional reinforcement learning algorithms fail to address. Agents have to deal with the lack of information about the environment's state and the opponents' beliefs and goals. A promising research direction is to learn models of the other agents to better understand their interactions. This project will investigate representation learning for opponent modelling in order to improve learning in multi-agent systems.
Arrasy Rahman (Sep 2018 – Apr 2023)
MSc Data Science, University of Edinburgh, 2017; BSc Computer Science, Universitas Indonesia, 2015
Project: Ad Hoc Teamwork in Open Multi-Agent Systems using Graph Neural Networks
Many real-world problems require an agent to achieve specific goals by interacting with other agents, without having predefined coordination protocols with other agents. Prior work on ad hoc teamwork focused on multi-agent systems in which the number of agents is assumed fixed. My project focuses on using Graph Neural Networks (GNNs) to handle interaction data between varying number of agents. We explore the possibility of combining GNNs with Reinforcement Learning techniques to implement agents that can perform well in teams with dynamic composition
Ibrahim H. Ahmed (Sep 2018 – May 2022)
MS in Computer Science, UC Davis, 2018; BS in Computer Science, UC Davis, 2016
Project: Quantum-Secure Authentication and Key Agreement via Abstract Multi-Agent Interaction
Authentication and key establishment are the foundation for secure communication over computer networks. However, modern protocols which rely on public key cryptography for secure communication are vulnerable to quantum technology–based attacks. My project studies a novel quantum-safe method for authentication and key establishment based on abstract multi-agent interaction. It introduces these fields to multi-agent techniques for optimisation and rational decision-making.
Visiting Researchers (Edinburgh)
Riccardo Zamboni (Oct 2024 – Feb 2025)
MSc Automation and Control Engineering, Polytechnic University of Milan 2019; BSc Industrial Engineering, University of Trento, 2017
Project: Offline Multi-Agent Reinforcement Learning
The sample inefficiency of many multi-agent reinforcement learning (MARL) algorithms presents considerable challenges for real-world applications. Unlocking the potential of offline methods in multi-agent settings is particularly compelling. Recent findings have also highlighted inconsistencies in baselines and evaluation protocols in offline MARL, revealing that simple independent learners can often compete with state-of-the-art offline algorithms. This research will focus on how to leverage information from other agents to surpass independent learning and foster effective coordination, even when dealing with small offline datasets.
Bram Renting (April 2024 – June 2024)
MSc Embedded Systems, Delft University of Technology, 2019; BSc Marine Technology, Delft University of Technology, 2016
Project: Multi-Agent Negotiation for Scheduling Human-Robot Cooperation in Warehouses
In warehouses where humans and robots are interdependent, precise coordination is required to reduce idle time and improve output. Traditional centralised coordination algorithms are complex and costly to design and do not transfer to other warehouses. Decentralised learning methods could alleviate some of the complexity and improve transferability. This project takes a multi-agent approach, where agents represent humans and robots. The agents plan routes to task locations and negotiate with other agents to schedule meetings to perform the task.
Xuehui Yu (July 2023 – June 2024)
BE Computer Science, Harbin Engineering University, 2019
Project: Generalisation in Reinforcement Learning via Causal Inference
Reinforcement Learning (RL) has proven to be an effective tool for training agents on difficult sequential decision-making problems. Most of the early successes focus on a fixed task in a fixed environment. However, in real applications, we often have changing environments, and RL agents may struggle to generalise well due to overfitting to their training environments. Such diverse, dynamic, and unpredictable environments make great demands on RL agents to reuse experience and adapt quickly. To improve generalization to unseen tasks, this project combines causal inference techniques with RL, enabling RL agents to understand the world from a causal perspective and quickly adapt to out-of-distribution domains.
Mahdi Kazemi Moghaddam (Dec 2022 – Feb 2023)
Hons. Degree of Bachelor of Computer Science, University of Adelaide, 2019; BSc Electrical Engineering, Electronics, Amirkabir University of Technology, 2015
Project: Fairness and Social Welfare in Autonomous Driving
Most of the current autonomous driving approaches based on reinforcement learning focus on maximising some notion of performance (e.g. travel time) or incorporating driving safety. As such, less attention has been paid to the social welfare and fairness aspects of the interaction of such trained policies with other road users. In this project, we aim to train a population of autonomous vehicles which can achieve high individual performance while cooperating with other agents to maximise the overall social welfare and fairness.
Alain Andres (June 2022 – Aug 2022)
MSc Telecommunication Engineering, University of the Basque Country, 2019; BSc Telecommunication Engineering, University of the Basque Country, 2017
Project: Exploration under Sparse Rewards with Deep Reinforcement Learning
Reinforcement learning algorithms are highly dependable on the feedback signals (rewards) provided by the environment. Unfortunately, the design of a suitable reward function is not always trivial and the adoption of sparse signals is a common choice to determine whether the task has been accomplished. However, such sparse reward signals lead to difficult exploration challenges. This project will investigate the adoption of Imitation Learning and Intrinsic Motivation solutions to enhance exploration and improve sample efficiency in challenging sparse-reward environments.
Giuseppe Vecchio (Feb 2022 – April 2022)
MSc in Computer Engineering, UNICT, 2020; BSc in Computer Engineering, UNICT, 2018
Project: Imitation Learning for Autonomous Robot Navigation in Unstructured Environments
Mobile robots have become part of everyday life in many forms like service robots, planetary rovers, autonomous cars and more. For ground robots the ability to navigate the surrounding environment is essential for long-term operations in real-world scenarios and is highly dependent on the ability to quickly adapt to new unseen settings. This project will explore the use of Imitation Learning-based techniques in simulation environments for autonomous navigation in unstructured settings and the use of domain adaptation approaches for real-world applications.
Maciej Wiatrak (Sep 2019 – Nov 2019)
BASc Mathematics & Computer Science, University College London, 2019
Project: Stabilising Generative Adversarial Networks with Multi-Agent Reinforcement Learning
Generative Adversarial Networks (GANs) are a state-of-the-art machine learning method, however, the training of GANs suffers from instability problems such as oscillatory behaviour and vanishing gradients. In this project, we outline the connections between GANs and Multi-Agent Reinforcement Learning (MARL), which is concerned with stable concurrent learning of multiple actors toward an equilibrium solution. We explore the connection between GANs and MARL, by proposing a GAN training method that utilises an established MARL technique based on variable learning rates.