Jacob Richard Whitehill

Jacob Richard Whitehill is an Associate Professor in the Department of Computer Science at Worcester Polytechnic Institute (WPI), with a research focus at the intersection of artificial intelligence and education. He leads a research group dedicated to applying machine learning, computer vision, and affective computing to educational contexts, with particular emphasis on classroom observation, student engagement, and intelligent tutoring systems. His work bridges multiple disciplines including cognitive science, psychology, and learning technologies.

Dr. Whitehill earned his BS in Computer Science from Stanford University (2001, Departmental Honors), MS in Computer Science from the University of the Western Cape (2007, Cum Laude), and PhD in Computer Science from the University of California, San Diego (2012). Prior to joining WPI, he served as a research scientist at Harvard University's Office of the Vice Provost for Advances in Learning and co-founded Emotient, a San Diego-based startup specializing in automatic emotion and facial expression recognition.

His research program centers on developing AI technologies that enhance educational experiences through automated classroom observation, student engagement analysis, and intelligent tutoring systems. Whitehill's work applies computer vision to track classroom dynamics, speech recognition to analyze student participation, and affective computing to understand emotional states during learning. His approach is highly interdisciplinary, integrating insights from educational psychology, cognitive science, and human-computer interaction to create practical educational technologies.

Whitehill's publication portfolio demonstrates a consistent trajectory toward increasingly sophisticated multimodal analysis of classroom environments. His recent work has shifted toward integrating multiple data streams (video, audio, text) to create comprehensive classroom analytics, with particular focus on equitable student participation, teacher-student interactions, and the development of AI systems that support rather than replace human educators. The research increasingly incorporates large language models while maintaining a strong foundation in traditional computer vision and machine learning techniques.

• NSF Grant: Developing New Scientific Instruments for Classroom Observation (AWD_ID=2046505)
• NSF AI Institute for Student-AI Teaming (iSAT)
• Schmidt Futures Grant: Hybrid Human-Agent Tutoring for Middle School Math
• $299,991 grant with Shichao Liu studying optimal indoor conditions for learning

As an advisor, Whitehill mentors several PhD students working on various aspects of AI in education, including classroom observation systems, speech processing for educational contexts, and multimodal learning analytics. His research is supported by significant external funding from NSF and other organizations, enabling his team to develop cutting-edge technologies that detect and boost student engagement. The lab collaborates extensively with institutions including the University of Colorado Boulder on AI education initiatives.

Whitehill leads a vibrant research group focused on applying machine learning to educational challenges, with ongoing projects spanning classroom observation technologies, AI teaching agents, and multimodal learning analytics. The group maintains strong connections with educational researchers and practitioners to ensure their technological innovations address real classroom needs.