Candidate Name: Benjamin Daniel Poole
Program:  Computing & Information Systems
Committee Chair: Dr. Minwoo Lee
Committee Members:  Dr. Min Shin, Dr. Xiang Zhang, Dr. Razvan Bunescu, Dr. Donald Jacobs
Abstract:  
Reinforcement learning (RL) algorithms have become more adept with each passing year, able to overcome increasingly complex problems. However, these gains have largely come through self-learning in fixed, stationary domains, with little consideration for whether agents act in ways consistent with human values. Addressing these limitations requires agents that align with human intentions and can adapt over time to both human input and shifting environments. Fragmented threads of work have begun to tackle these challenges separately: interactive RL incorporates human input to help agents stay aligned, while continual RL enables agents to adapt to environment changes and retain prior knowledge. Together, these threads motivate the concept of a shapeable agent, an idealized agent that learns continuously through a combination of self-learning and multimodal human input. This thesis proposes a novel multimodal algorithm for alignment through human input, without loss of performance. A promising new feedback medium, intrinsic feedback, is then explored, examining both its viability and how it compares to more realistic, non-expert forms of feedback when integrated with the proposed algorithm. Finally, data rehearsal is investigated as a means of adapting value-function algorithms to improve continual learning in more realistic, multi-cyclic domain shift settings. Each contribution addresses a piece of this larger puzzle, advancing the broader realization of a shapeable agent.