Meta Research

We propose to combine reinforcement learning and theoretical physics to describe effective theories of agency. This involves understanding the connection between the physics notion of causality and how intelligent agents can arise as a useful effective description within some environments. We discuss cases where such an effective theory of agency can break down and suggest a broader framework incorporating theory of mind for expanding the notion of agency in the presence of other agents that can predict actions. We comment on implications for superintelligence and whether physical bounds can be used to place limits on such predictors.