Reinforcement Learning, Logic and Evolutionary Computation: A Learning Classifier System Approach to Relational Reinforcement Learning

by Drew Mellor

Reinforcement learning (RL) consists of methods that automatically adjust behaviour based on numerical rewards and penalties. While use of the attribute-value framework is widespread in RL, it has limited expressive power. Logic languages, such as first-order logic, provide a more expressive framework, and their use in RL has led to the field of relational RL. This thesis develops a system for relational RL based on learning classifier systems (LCS). In brief, the system generates, evolves, and evaluates a population of condition-action rules, which take the form of definite clauses over first-order logic. Adopting the LCS approach allows the resulting system to integrate several desirable model-free and "tabula rasa" learning; a Markov Decision Process problem model; and importantly, support for variables as a principal mechanism for generalisation. The utility of variables is demonstrated by the system's ability to learn genuinely scalable behaviour - behaviour learnt in small environments that translates to arbitrary large versions of the environment without the need for retraining.

292 pages, Paperback

Published May 15, 2010