SODA: Self-Organizing Distinctive State Abstraction
A major current challenge in reinforcement learning research is to extend methods that work well on discrete, short-range, low-dimensional problems to continuous, highdiameter, high-dimensional problems, such as robot navigation using high-resolution sensors. Using SODA an robot in a continuous world can, with little prior knowledge of its sensorimotor system, environment, and task, improve task learning by first using a self-organizing feature map to develop a set of higher-level perceptual features while exploring using primitive, local actions. Then using those features, the agent can build a set of high-level actions that carry it between perceptually distinctive states in the environment. SODA combines a perceptual abstraction of the agent’s sensory input into useful perceptual features, and a temporal abstraction of the agent’s motor output into extended, high-level actions, thus reducing both the dimensionality and the diameter of the task.