Ph.D. Dissertation

Causality-Inspired Reinforcement Learning:
State Abstractions, Exploration, and Representations

Zizhao Wang, The University of Texas at Austin, 2026
Supervisor: Peter Stone

Abstract

Reinforcement learning offers a versatile paradigm for developing autonomous decision-making agents, but many current algorithms still require large amounts of data and generalize poorly. One central difficulty is that correlation-based learning can entangle all observed state factors with actions, increasing sample complexity and making learned policies vulnerable to spurious correlations.

This dissertation studies how causal reasoning can improve the sample efficiency and generalization of RL algorithms. Through the lens of causality, an agent can reason about which actions and state factors affect future states, and which factors determine task success. These structures support more accurate dynamics and reward models, more compact state abstractions, strategic exploration, reusable skill discovery, and structured representations for low-level observations.

The thesis contributes methods for learning minimal causal state abstractions, designing intrinsic rewards from local causal dependencies, discovering reusable skills that generate meaningful factor interactions, and extracting structured state and action representations when high-level factors are not directly available. Together, these contributions help agents infer the causes and consequences of their actions, generalize to unseen states, and learn new tasks with limited data.