CPS Events
Reinforcement Learning for Large-Scale Games
Abstract
This talk addresses the use of reinforcement learning in two-player zero-sum Markov games with finite but large state spaces, for which the goal is to find minimax policies with “modest”' computation. We use the qualifier “modest” to mean that we seek to certify policies as optimal without exploring the full state-space of the game.
The approach followed is strongly motivated by Q-learning, which was proposed in the late 1980s to extend the single-player dynamic programming principle to model-free reinforcement learning by eliminating the need for a known transition model. Extensions of Q-learning to two-player zero-sum games appeared shortly after. Since then, most of the work devoted to proving correctness of Q-learning relies on establishing that its iteration converges to a unique fixed-point of a Bellman-like equation, which generally requires exploring the full state-space.
We will see that, for zero-sum games, it is possible to construct provably correct optimal policies using algorithms inspired by Q-learning, without requiring convergence of the Q function over the whole state-space. In fact, the samples used to update the Q-function may not even explore the whole set of reachable states and, for certain classes of games, the fraction of explored states gets smaller and smaller as the size of the state-space increases.
Speaker Bio
João Pedro Hespanha received his Ph.D. degree in electrical engineering and applied science from Yale University, New Haven, Connecticut in 1998. From 1999 to 2001, he was Assistant Professor at the University of Southern California, Los Angeles. He moved to the University of California, Santa Barbara in 2002, where he currently holds a Distinguished Professor position with the Department of Electrical and Computer Engineering.
Dr. Hespanha is a Fellow of the International Federation of Automatic Control (IFAC) and of the IEEE. He was an IEEE distinguished lecturer from 2007 to 2013. His current research interests include multi-agent control systems; game theory; optimization; distributed control over communication networks (also known as networked control systems); stochastic modeling in biology; and network security. Additional information about his research and publications available at https://web.ece.ucsb.edu/~hespanha/.
Human Acceptance of Autonomous Systems
Abstract
This seminar explores how society engages with autonomous transportation systems, focusing on automated vehicles and Advanced Air Mobility (AAM). Dr. Sina Nordhoff will present research on human acceptance, trust, and safety, emphasizing that public confidence and social readiness are essential alongside technological progress. Drawing on theoretical models, real-world applications, and extensive empirical data, including over 220 interviews and 40,000 surveys, Dr. Nordhoff will identify key factors shaping acceptance, such as socio-demographics, personality traits, perceived risks and benefits, and the effects of misuse or miscalibrated trust. The seminar will highlight how ethical considerations, societal norms, and regulatory frameworks influence deployment. Attendees will gain insight into how this work can guide policymakers, industry, and communities in ensuring responsible, equitable, and safe implementation. Dr. Nordhoff will also briefly discuss future research directions.
Speaker Bio
Dr. Sina Nordhoff is a leading expert in the field of human factors and user acceptance of new and emerging transportation technologies. She holds a Ph.D. from Delft University of Technology and is affiliated with the University of California, Davis. Dr. Nordhoff specializes in electric vehicles and automated vehicles (AVs), focusing on how to responsibly integrate these innovations into society. Her research spans theoretical models, empirical studies, and real-world applications, involving over 220 interviews and 40,000 analyzed surveys. She has developed innovative frameworks to understand human acceptance, trust, and safety, addressing critical issues such as misuse, trust miscalibration, and cyber-physical attacks. Dr. Nordhoff's research is published in top-tier journals and has garnered significant attention from policymakers and industry leaders. Her work aims to inform the design, deployment, and regulation of these technologies to ensure they are safe, equitable, and socially beneficial. Dr. Nordhoff's current research agenda includes pioneering efforts in interdisciplinary theory development, safety assessment, and understanding cognitive measurements. Her overarching goal is to bridge the gap between technological advancements and societal well-being, creating a future where transportation benefits all members of society.
Robots that Know What They Do Not Know: Assured AI-enabled Autonomy in Unknown Environments
Abstract
Designing robots that navigate unfamiliar environments to execute natural language (NL) commands is a cornerstone of advanced embodied intelligence. While recent AI-enabled architectures have made impressive empirical progress, they often lack introspection, leading to systems that act with unwarranted confidence, unaware of their own limitations or whether they have successfully completed their tasks. As a result, these systems offer limited performance and safety guarantees, restricting their deployment in safety-critical settings.
In this talk, I will present an introspective, neuro-symbolic autonomy architecture that enables robots to complete NL tasks in unknown environments with assurance guarantees by explicitly quantifying their own uncertainty using uncertainty quantification (UQ) tools. The neural component employs large language models (LLMs) to translate NL commands into temporal logic specifications, while leveraging conformal prediction, a UQ tool, to calibrate and quantify prediction uncertainty arising from LLM imperfections and potential NL ambiguity. When uncertainty exceeds user-defined thresholds, uncertainty-aware feedback is solicited from auxiliary LLMs—or, if necessary, from human operators. We provide theoretical guarantees, supported by empirical case studies, that the proposed uncertainty-aware translation framework, called ConformalNL2LTL, achieves user-specified translation success rates under certain distributional settings. The symbolic component generates plans for mobile robots with AI-enabled perception systems to satisfy temporal logic tasks while explicitly reasoning over perceptual and environmental uncertainty. This allows robots to decide when to proceed confidently and when to actively gather additional sensor data, ensuring task completion with the desired probability. Notably, the developed planners are agnostic to specific sensor models or noise characteristics. The talk will conclude with case studies and demonstrations, followed by a discussion of limitations and open problems.
Speaker Bio
Yiannis Kantaros is an Assistant Professor in the Department of Electrical and Systems Engineering, Washington University in St. Louis (WashU), St. Louis, MO, USA. He earned a Diploma in Electrical and Computer Engineering in 2012 from the University of Patras, Greece, and M.Sc. and Ph.D. degrees in Mechanical Engineering from Duke University, Durham, NC, in 2017 and 2018, respectively. Prior to joining WashU, he was a postdoctoral associate in the Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA. His current research interests include machine learning, distributed control and optimization, and formal methods with applications in robotics. He received the Best Student Paper Award at the 2nd IEEE Global Conference on Signal and Information Processing (GlobalSIP) in 2014 and was a finalist for the Best Multi-Robot Systems Paper at the IEEE International Conference on Robotics and Automation (ICRA) in 2024 and a finalist for the Best Paper Award at the ACM/IEEE International Conference on Cyber-physical Systems (CPSWeek-ICCPS) in 2025. He also received the 2017-18 Outstanding Dissertation Research Award from the Department of Mechanical Engineering and Materials Science at Duke University and a 2024 NSF CAREER Award.
Some Thoughts and Ideas on the Quest for Safe Autonomy of Aerospace Vehicles
Abstract
Autonomous aerospace vehicles have applications in diverse fields like space exploration, disaster monitoring, infrastructure inspection, and transportation. However, in spite of several years of research on autonomy of aerospace vehicles, substantial challenges remain in achieving safe and reliable autonomy. The biggest challenges in integrating autonomous aerospace vehicles into human society have to do with dynamic uncertainties due to natural (environmental) effects and safety and reliability in interactions with humans. Recent advances in learning-based and data-enabled control and navigation have made it possible to deal with these challenges to some extent. However, most of these schemes lack nonlinear stability and robustness to external inputs or internal unknowns, which are needed to meet the stringent requirements of safety and reliability for autonomous aerospace vehicles. This talk presents some of my thoughts and ideas on safe autonomous operations of aerospace vehicles, based on how my research has handled these challenges. A key feature of this research is the stress on nonlinear stability and robustness of GNC algorithms for autonomous aerospace vehicles in the presence of actuator constraints, sensor and onboard processor capabilities, and dynamic (time-varying) uncertain inputs or disturbances. Future directions that include learning-based control for autonomous aerospace vehicles are also touched upon.
Speaker Bio
Amit Sanyal obtained the B.Tech. degree in Aerospace Engineering from the Indian Institute of Technology, Kanpur, in 1999. He completed his MS in Aerospace Engineering from Texas A&M University in 2001, where he received the Distinguished Graduate Student Master's Research Award. He obtained his Ph.D. in Aerospace Engineering and MS in Mathematics from the University of Michigan in 2004 and 2005, respectively, and was the recipient of an Engineering Academic Scholar Certificate. After his post-doctoral research at Arizona State University in 2005–2006, he joined the faculty in Mechanical Engineering at the University of Hawaii in 2007. He has been a faculty member in Mechanical and Aerospace Engineering at New Mexico State University (2010–2015), and is currently a faculty member in Mechanical and Aerospace Engineering at Syracuse University. He develops and applies techniques from geometric mechanics, nonlinear and geometric control, and continuous and discrete-time Lagrangian/Hamiltonian systems, to dynamics modeling, guidance, navigation, and control of unmanned and autonomous systems. He is an associate fellow of AIAA, a senior member of IEEE, and a member of ASME and SIAM. His research has been supported by NSF, NASA, and AFOSR, and he co-founded Akrobotix LLC, a robotics start-up company.
Cyber-Physical Systems in Agriculture: Integrating Soft Robotics and Human-in-the-Loop Mechanisms
Abstract
Tree fruit growers worldwide are grappling with labor shortages in crucial operations such as harvesting and pruning. The development of robotic solutions for these labor-intensive tasks has generated significant interest. However, existing efforts have proven to be excessively expensive, slow, or necessitate orchard reconfiguration for functionality.
In this presentation, I introduce three alternative approaches to agriculture robot development: 1) Utilizing a cyber-physical system that facilitates human participation and machine learning for collaborative robotic apple harvesting. 2) Introducing a novel soft-growing multi-robot platform to reduce costs and enhance overall responsiveness. 3) Learning human-robot collaboration behaviors to optimize efficiency.
Leveraging human-robot interaction to address existing individual shortcomings of humans and robots will enable mutual learning of behaviors to enhance overall efficiency.
Speaker Bio
Dr. Ming Luo is currently a Flaherty Assistant Professor in the School of Mechanical and Materials Engineering at Washington State University. He is the director of the Mechanically-Intelligent Autonomous Robotics (MIAR) Laboratory. From 2018 to 2020, he was a postdoctoral scholar in the Department of Mechanical Engineering at Stanford University. He received his Ph.D. in Robotics Engineering from Worcester Polytechnic Institute in 2017. His academic interests include agricultural robotics, soft robotics, snake robots, origami robots, and haptics. His research has been funded by the NSF, USDA, NIFA, and the Washington Tree Fruit Research Commission. His work has also been featured in The Wall Street Journal, IEEE Xplore, Reuters, and The Verge.
