Using Reinforcement Learning to Control Life Support Systems 2004-01-2439

Advanced life support systems have many interacting processes and limited resources. Controlling and optimizing advanced life support systems presents unique challenges that are addressed in this paper. We have developed a controller using reinforcement learning [Barto&Sutton], which actively explores the space of possible control strategies, guided by rewards from a user specified long term objective function. We evaluated this controller using Biosim, our discrete event simulation of an advanced life support system. This simulation supports all life support modules including crew, food production, air revitalization, water recovery, solid waste incineration and power. Our algorithm for reinforcement learning discovered unobvious strategies for maximizing mission length. By exploiting nonlinearities in the simulation dynamics, the learned controller outperforms a controller designed by an expert.