University of Minnesota
Computer Science & Engineering
http://www.cs.umn.edu/

Taming Latency Variability and Scaling Deep Learning

Cray Distinguished Speaker Series

Thursday, November 07, 2013

Presenter: Jeff Dean
Affiliation: Google
Time: 13:00 - 14:00
Location: Keller Hall 3-180
Host: Vipin Kumar
Schedule:View Extended Schedule Details

 

Today's large-scale web services operate in warehouse-sized datacenters and run on clusters of machines that are shared across many kinds of interactive and batch jobs.

 In the first part of this talk, I'll describe a collection of techniques and practices for lowering response times (especially in the tail of the latency distribution) in large distributed systems whose components run on shared clusters of machines, where pieces of these systems are subject to interference by other tasks, and where unpredictable latency hiccups are the norm, not the exception.

 In the second part of the talk, I'll highlight some recent work on using large-scale distributed systems for training deep neural networks.  I'll discuss how we can utilize both model-level parallelism and data-level parallelism in order to train large models on large datasets more quickly.  I'll also highlight how we have applied this work to a variety of problems in domains such as speech recognition, object recognition, and language modeling.

 Bio:

 Jeff joined Google in 1999 and is currently a Google Fellow in Google's Knowledge Group.  He has co-designed/implemented five generations of Google's crawling, indexing, and query serving systems, and co-designed/implemented major pieces of Google's initial advertising and AdSense for Content systems.  He is also a co-designer and co-implementor of Google's distributed computing infrastructure, including the MapReduce, BigTable and Spanner systems, protocol buffers, LevelDB, systems infrastructure for statistical machine translation, and a variety of internal and external libraries and developer tools.  He is currently working on large-scale distributed systems for machine learning.  He is a Fellow of the ACM, a member of the U.S. National Academy of Engineering, and a recipient of the Mark Weiser Award and the ACM-Infosys Foundation Award in the Computing Sciences.

 

 

Contact CS&E | CS&E Employment | Site Map
Contact: 4-192 Keller Hall, 200 Union St, Minneapolis, MN 55455     Phone: (612) 625-4002