Bringing Digital Empathy to Conversational AI Agents
Empathy is central to everyday interactions and enables stronger collaboration, leadership, and communication. Extending this human capacity to conversational AI can raise engagement and satisfaction, but it also brings risks such as dependency and overreliance. This talk shares recent efforts at Microsoft Research to map the many forms of digital empathy and highlights the opportunities and challenges of building it into conversational agents.

About our speaker
Javier Hernandez is a Principal Researcher in the Interactive Multimodal Futures group at Microsoft Research and an Associate Editor of the IEEE Transactions on Affective Computing journal. Javier’s research focuses on how future AI agents can collaborate effectively by understanding users more deeply and adapting to their context. Before Microsoft, he was a Research Scientist at the MIT Media Lab, where he earned his Ph.D. in the Affective Computing group. His work, spanning mental health support, emotion-aware technology, and user-centered AI, has received multiple best paper awards and has been featured in National Geographic and The Economists.

Naviagation: Enter the building through this gate and take left.

Nearest T station is Ruggles on Orange line and Northeastern University on Green line

LLMs as UXR Participants?: A How-to Guide and Comparative Analysis
This talk explores the potential and limitations of using Large Language Models (LLMs) as surrogate research participants through a series of simulated choice-based survey experiments. The first half details an open-source Python program I built that runs Maximum Difference Scaling (MaxDiff) experiments—a survey method where participants choose the most and least important items from sets of options—using LLM users, including customizable personas and comprehensive analytics reporting. The talk will walk through the AI-assisted development process, laying out best practices for AI-assisted software development, covering key considerations like building in stages, implementing unit tests, enforcing structured LLM outputs, and managing API costs effectively.

The second half describes the methods and findings of an experiment using this application. By comparing a large sample of LLM-generated personas against real data from humans, I demonstrate that LLMs can achieve moderate alignment with aggregate human preferences but fundamentally fail to capture human variability, even at maximum temperature settings. Most strikingly, removing a single seemingly-innocuous sentence from the system prompt completely reshuffled individual model-human alignment while leaving aggregate alignment relatively unchanged. These findings reveal the stark and often unpredictable sensitivity of LLM models to prompt engineering, an effect that may be moderated by model temperature. These findings have important implications for responsible AI and user research applications. As we increasingly rely on AI for understanding human needs and preferences, it is critical to recognize that subtle prompt variations can alter research outcomes in unpredictable ways, with the potential to amplify or obscure bias baked into LLMs and underscoring the need for rigorous prompt testing and evaluation.

About our speaker
Dr. Aaron Gardony was a Cognitive Scientist at the DEVCOM Soldier Center and a Visiting Scientist at the Center for Applied Brain and Cognitive Sciences (CABCS) at the time of this work. He received his joint doctorate in Psychology and Cognitive Science from Tufts University in 2016, a Master of Science from Tufts University in 2014, and a BA from Tufts University in 2009. His current work focuses on Responsible AI and Safety Evaluation.

Naviagation: Enter the building through this gate and take left.