Hello, I'm Yi Fei!
I am a Ph.D. student at the Human-Computer Interaction Institute in Carnegie Mellon University's School of Computer Science, advised by David Lindlbauer.
Computing is becoming increasingly integrated into everyday life, shaping how people communicate, access information, and experience the physical world. Situated in the real world, computing must contend with the complexity of human behavior, social dynamics, and physical environments, creating both new opportunities and challenges for the design of interactive systems. My research focuses on situated interaction, asking: How can we build systems that seamlessly augment human capabilities in situ?
My work explores this question across a range of technologies, such as conversational agents, robots, and Extended Reality (XR), through two complementary threads:
- Situated agentic systems. I build and study LLM-based agents that operate within people's physical and social contexts, such as agents that feel present in the room through spatial audio (CHI 2026) and agents that respond on users' behalf in voice calls (CHI 2025).
- Human behavior models for situated interaction. I develop model-based techniques that adapt spatial interfaces to users' environments and interactions (UIST 2021, UIST 2023), and learning-based methods that predict whether users notice ambient information (CHI 2025). Most recently, I have been betting on user simulation to model, evaluate, and design for situated interaction at scale (EMNLP 2026).
My work has been published in leading venues in HCI, XR, and NLP, including CHI, UIST, TVCG, and EMNLP, and has received CHI Best Paper and Honorable Mention awards. My research has been supported by the Croucher Foundation Scholarship and sponsored research from the Honda Research Institute, Fujitsu, and Sightful. Previously, I held research positions at Microsoft Research, Google, and Sony Computer Science Laboratories. I received my B.A. in Computer Science from Swarthmore College.
CV | Github | Google Scholar | LinkedIn | Email
Publications
- 2026
- Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations Conference on Empirical Methods in Natural Language Processing (EMNLP) 2026 · Oral [Paper]
- Auditorily Embodied Conversational Agents: Effects of Spatialization and Situated Audio Cues on Presence and Social Perception ACM CHI Conference on Human Factors in Computing Systems 2026 [Paper]
- One Body, Two Minds: Alternating VR Perspective During Remote Teleoperation of Supernumerary Limbs ACM CHI Conference on Human Factors in Computing Systems 2026 [Paper]
- 2025
- Augmented Reality Productivity In-the-Wild: A Diary Study of Usage Patterns and Experiences of Working With AR Laptops in Real-World Settings IEEE Transactions on Visualization and Computer Graphics (TVCG) 2025 [Paper]
- Sensing Noticeability in Ambient Information Environments ACM CHI Conference on Human Factors in Computing Systems 2025 [Paper]
- Conversational Agents on Your Behalf: Opportunities and Challenges of Shared Autonomy in Voice Communication for Multitasking ACM CHI Conference on Human Factors in Computing Systems 2025 · Best Paper Award [Paper]
- 2024
- First or Third-Person Hearing? A Controlled Evaluation of Auditory Perspective on Embodiment and Sound Localization Performance IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2024 [Paper]
- New Ears: An Exploratory Study of Audio Interaction Techniques for Performing Search in a Virtual Reality Environment IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2024 [Paper]
- Predicting the Noticeability of Dynamic Virtual Elements in Virtual Reality ACM CHI Conference on Human Factors in Computing Systems 2024 [Paper]
- MARingBA: Music-Adaptive Ringtones for Blended Audio Notification Delivery ACM CHI Conference on Human Factors in Computing Systems 2024 · Honorable Mention Award [Paper]
- 2023
- InteractionAdapt: Interaction-driven Workspace Adaptation for Situated Virtual Reality Environments ACM Symposium on User Interface Software and Technology (UIST) 2023 [Paper]
- HOOV: Hand Out-Of-View Tracking for Proprioceptive Interaction using Inertial Sensing ACM CHI Conference on Human Factors in Computing Systems 2023 [Paper]
- Cross-Device Shortcuts: Seamless Attention-guided Content Transfer via Opportunistic Deep Links between Apps and Devices ACM International Conference on Multimodal Interaction (ICMI) 2023 [Paper]
- Controllers or Bare Hands? A Controlled Evaluation of Input Techniques on Interaction Performance and Exertion in Virtual Reality IEEE Transactions on Visualization and Computer Graphics (TVCG) 2023 [Paper]
- 2022
- Towards Understanding Diminished Reality ACM CHI Conference on Human Factors in Computing Systems 2022 [Paper]
- XSpace: An Augmented Reality Toolkit for Enabling Spatially-Aware Distributed Collaboration ACM Interactive Surfaces and Spaces Conference (ISS) 2022 [Paper]
- ComforTable User Interfaces: Surfaces Reduce Input Error, Time, and Exertion for Tabletop and Mid-air User Interfaces IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2022 [Paper]
- 2021
- SemanticAdapt: Optimization-based Adaptation of Mixed Reality Layouts Leveraging Virtual-physical Semantic Connections ACM Symposium on User Interface Software and Technology (UIST) 2021 [Paper]
- Hybrid Paper-Digital Interfaces: A Systematic Literature Review ACM Designing Interactive Systems Conference (DIS) 2021 [Paper]
- XRStudio: A Virtual Production and Live Streaming System for Immersive Instructional Experiences ACM CHI Conference on Human Factors in Computing Systems 2021 [Paper]
Art
As an undergraduate, I spent a significant amount of time painting and drawing. I was one course shy of a second major in Studio Art before deciding to graduate early and work with Christian Holz at ETH Zurich instead. I still think fondly of my Beardsley studio and credit much of my identity as a researcher to my training as an artist with Logan Grider and Randall Exon.
Drawings
Paintings
