Colloquium: 3D Behavioral Imaging and Beyond

Colloquia
October 22, 2018 - 11:15am to 12:30pm
Affiliation: 
CS&E UMN
Location: 
Mechanical Engineering 212
Abstract: Nonverbal social signals such as gaze direction, facial expression, and body gesture have been ingrained into our interpersonal communications, which often appear at microscopic scale. Despite their omnipresence in all aspects of social interactions, existing AI systems are nearly blinded to them. In this talk, I will walk through our effort towards enabling 3D behavioral imaging---a computational model that allows precise measurements of microscopic social signals from numerous multiview cameras. A key challenge is that social interactions inherently induce self-occlusion, which fundamentally limits accurate 3D reconstruction from the image streams. I will argue that associating semantic meaning with geometry, e.g., holistic finger pose, provides a strong cue to predict the missing data. To learn such visual semantics, I will introduce a new large-scale human behavior dataset called PublicHumans scanned by 107 HD cameras at Minnesota State Fair. In the second part of the talk, I will discuss a computational approach to measure free-ranging behaviors of monkeys for neuroscience study. Unlike humans, these monkeys are challenging due to lack of annotation data. To address this, I will introduce a semi-supervised learning framework that leverages multiview geometry and tracking to reconstruct their motion in 3D.
Bio: Hyun Soo Park is an Assistant Professor at the Department of Computer Science and Engineering, the University of Minnesota. He is interested in computer vision approaches for behavioral imaging. Prior to the UMN, he was a Postdoctoral Fellow in GRASP Lab at University of Pennsylvania. He earned his Ph.D. from Carnegie Mellon University.