Using Generative AI to Produce Situated Action Recommendations in Augmented Reality for High-level Goals

Generative AI and HCI (GenAICHI) Workshop at CHI


Many people pursue long-term high-level goals related to areas such as fitness, mental health, or skill enhancement. Additionally, each individual pursuing a high-level goal may pursue it differently than others, depending on their contexts. In this research, we explore the use of generative AI for producing context-dependent and situated recommendations in augmented reality (AR) for actions that individuals can take toward their high-level goals. We developed a technology probe and ran a user study in a mock home environment to understand how users experience and perceive such recommendations in the context of a home environment and the tools contained within it. We found that users value the passive nature of such recommendations, and the roles that they can play to inform, motivate, and inspire the user. However, compared to receiving advice from friends, family, and experts, users are still skeptical about AI-generated suggestions, particularly for critical and sensitive goals, and there are still design considerations to explore to understand how to improve the content and delivery of such recommendations.

Featured Publications