Affective Computing:Emotion, AI, and Human Behavior

CS 3891 / 5891 โ€ข MWF 11:15 AMโ€“12:05 PM

How can AI understand and respond intelligently to human emotion?

This interdisciplinary, project-based course brings together ideas from artificial intelligence, psychology, and human-centered design to explore how emotion shapes human behavior and how computational systems can recognize, model, and influence it.

View a Previous Course Syllabus โ†’
Human and artificial agent connected by emotion signals
Project-Based Learning

Build something that matters

Working in interdisciplinary teams, students identify a real-world problem, propose and pitch an affective computing solution, build a prototype using existing AI tools, and design an evaluation to determine whether it works.

๐Ÿ† Several student projects have led to peer-reviewed publications at major conferences including NAACL, AAAI, ACII, IVA, and others.
๐Ÿ’ก
1. Identifya real-world problem
๐Ÿ‘ฅ
2. Forminterdisciplinary teams
๐Ÿ’ป
3. Builda prototype with AI tools
๐Ÿ“Š
4. Evaluateand measure impact

Background / Prerequisites

Students should have prior preparation through one of two routes.

๐Ÿ’ป Computer Science Students

Students should have substantial programming experience (e.g., CS 2201, CS 2204 or equivalent); prior coursework in artificial intelligence or machine learning is recommended.

๐Ÿง  Psychology / Cognitive Science Students

Students should have completed an introductory statistics course (e.g., PSY 2100 or PSY-PC 2110) and a course in experimental design or research methods (PSY 2150, PSY-PC 2170 or PSY-PC 2180, or equivalent). More advanced statistics coursework (e.g., PSY-PC 2120) would be beneficial.

From Course Project to Publication

Selected Publications from Student Projects

Course projects can become genuine research. These examples show how student work has developed into peer-reviewed publications.

NAACL 2024

Can language model moderators improve the health of online discourse?

Hyundong Cho et al.

NAACL-HLT 2024
IVA 2025

Can LLMs Generate Behaviors for Embodied Virtual Agents Based on Personality Prompting?

Bin Han, Brian Kwon, Spencer Lin, Kaleen Shrestha, Jonathan Gratch

25th ACM International Conference on Intelligent Virtual Agents
AAAI 2026

Can LLMs Truly Embody Human Personality? Analyzing AI and Human Behavior Alignment in Dispute Resolution.

Deuksin Kwon, Kaleen Shrestha, Bin Han, Spencer Lin, James Anthony Hale, Jonathan Gratch, Maja Mataric, Gale Lucas

AAAI 2026
ACII 2023

Context unlocks emotions: Text-based emotion classification dataset auditing with large language models.

Daniel Yang et al.

ACII 2023
Haptics 2024

How Visualizing Touch Can Transform Perceptions of Intensity, Realism, and Emotion?

Xin Zhu et al.

Human Haptic Sensing and Touch Enabled Computer Applications, 2024
IVA 2011

Virtual Rapport 2.0

Lixing Huang, Louis-Philippe Morency, Jonathan Gratch

International Workshop on Intelligent Virtual Agents, 2011
View additional publications from course projects
  • Carlos Busso, Sungbok Lee, and Shrikanth S. Narayanan. โ€œUsing neutral speech models for emotional speech analysis.โ€ Interspeech, 2007.
  • Jangwon Kim, Sungbok Lee, and Shrikanth S. Narayanan. โ€œDetailed study of articulatory kinematics of critical articulators and non-critical articulators of emotional speech.โ€ 162nd Meeting of the Acoustical Society of America, 2011.
  • Jillian Greczek, Katelyn Swift-Spong, and Maja Matariฤ‡. Using eye shape to improve affect recognition on a humanoid robot with limited expression. Technical report, 2011.

What Have Students Built?

Past projects have explored a wide range of applications across emotion, AI, interaction, media, and human behavior.

๐ŸŽ™ Emotion and Speech

  • End-to-end speech negotiations
  • Emotionally-adapted automatic speech recognition
  • Emotion Mirror

๐ŸŽฎ Games and Motivation

  • Emotion-based reinforcement learning
  • Regulatory fit and game performance
  • Extrinsic rewards and intrinsic motivation

๐Ÿค Social Interaction

  • Real-time engagement analysis
  • Emotion modeling for explanations
  • Embodied virtual agents

๐ŸŽต Music and Media

  • Popular music and emotion
  • Empathetic music player
  • Emotional Quake

๐Ÿ‘ Perception and Sensing

  • Vision-assisted animation
  • Visualizing touch and haptics
  • Affect recognition

โš™ Human-Centered AI

  • AI personality and behavior generation
  • Dispute resolution and human behavior
  • Principled model of self-efficacy

Interested in the course?

See a previous offering for a detailed example of topics, readings, assignments, and course structure.

View a Previous Course Syllabus โ†’