Colloquium: Adapting CNNs to the Dynamic 3D World

April 12, 2019 - 11:15am to 12:15pm
Deqing Sun
3-180 Keller Hall
Hyun Soo Park
Abstract: Significant progress has been made on 2D image understanding by convolutional neural networks (CNNs), trained on huge amounts of labeled data. However, the goal of enabling intelligent machines to exist in and interact with the dynamic 3D world remains elusive. To this end, I argue that we need to build domain knowledge into CNNs and reduce their reliance on labeled data. In this talk, I will present my recent work in these directions, including, 1) building classical principles of optical flow into the CNN architecture; 2) exploring self-supervision signals to learn optical flow; 3) designing sparse lattice networks for point cloud processing. All demonstrate the benefits of combining classical vision reasoning with deep learning. 
Bio: Deqing Sun is a senior research scientist at NVIDIA. He received the Ph.D. degree in Computer Science from Brown University under the supervision of Prof. Michael J. Black, the M.Phil. degree in Electronic Engineering from the Chinese University of Hong Kong, and the B.Eng. degree in Electronic and Information Engineering from Harbin Institute of Technology. He was a research intern at Microsoft Research New England in 2010, mentored by Dr. Ce Liu. He was a postdoctoral research fellow and is a visiting researcher at Harvard University, working with Prof. Hanspeter Pfister. His research interests include computer vision and machine learning, and he has focused on optical flow estimation and its applications to videos in the past eleven years. He served as an area chair for ECCV 2018, CVPR 2019, and BMVC 2019, and co-organized “what is optical flow for?” workshop at ECCV 2018 and “deep learning for content creation” tutorial at CVPR 2019. He and his collaborators won the first place in the optical flow competition of the robust vision challenge and received a best paper honorable mention award at CVPR 2018 for their work on sparse lattice networks for point cloud processing.