Cray Colloquium: 3D Scene Understanding with a RGB-D Camera

Cray Distinguished Speaker Series
October 14, 2019 - 11:15am to 12:15pm
Thomas Funkhouser

Abstract: Three-dimensional scene understanding is important for computer systems
that respond to and/or interact with the physical world, such as robotic
manipulation, autonomous navigation, and augmented reality.  For
example, they may need to estimate the 3D geometry of the surrounding
space (e.g., in order to navigate without collisions) and/or to
recognize the semantic categories of nearby objects (e.g., in order to
interact with them appropriately).   In this talk, I will describe
recent work on 3D scene understanding at Princeton and Google.   I will
focus on three projects that infer 3D structural and semantic models of
scenes from partial observations with a RGB-D camera.   The first learns
to infer 3D geometry from color in cases where the depth sensing is not
possible.  The second learns to infer the 3D structure and semantics of
a complete surrounding environment.  The third infers a temporal model
of semantic object instances from RGB-D scans acquired at sparse time
intervals.   For each project, I will discuss the problem formulation,
scene representation, network architecture, dataset curation, and
potential applications.

This is joint work with Angela Dai, Maciej Halber, Jingwei Huang,
Matthias Niessner, Shuran Song, Andy Zeng, and Yinda Zhang.

Bio: Thomas Funkhouser is a senior research scientist at Google and the David
M. Siegel Professor of Computer Science, Emeritus, at Princeton
University.  He received a PhD in computer science from UC Berkeley in
1993 and was a member of the technical staff at Bell Labs until 1997
before joining the faculty at Princeton.  For most of his career, he
focused on research problems in computer graphics, including
foundational work on 3D shape retrieval, analysis, and modeling.   His
most recent research has focused on 3D scene understanding in computer
vision.   He has published more than 100 research papers and received
several awards, including the ACM SIGGRAPH Computer Graphics Achievement
Award, ACM SIGGRAPH Academy, ACM Fellow, NSF Career Award, Emerson
Electric, E. Lawrence Keyes Faculty Advancement Award, Google Faculty
Research Awards, University Council Excellence in Teaching Awards, and a
Sloan Fellowship.