Goal Inference using Reward-Producing Programs in a Novel Physics Environment

This academic paper investigates how humans represent and infer goals, proposing that goals can be formalized as reward-producing programs. The researchers developed a physics-based game environment where participants created and demonstrated novel goals. By collecting natural language descriptions and formal scoring criteria, they analyzed the relationship between goal complexity, reward structure, and perceived difficulty. A proof-of-concept computational method is presented, demonstrating the potential to infer a participant's goal by analyzing the trajectories of their in-game actions and comparing them against possible program-based goal representations. This research contributes to understanding human cognition and developing more sophisticated AI agents capable of generating and understanding novel objectives.