Outobox - Machine Learning Reading Group

This is an interdisciplinary group of faculty (Arindam Banerjee, Dan Boley, Maria Gini, Rui Kuang, Paul Schrater, and William Schuler) and students who meet every week to discuss research ideas and to share relevant literature. The group is called Outobox, because we like to think ...out of the box. There is a mailing list associated with this group. If you would like to be added, please go to sign-up page.

There is a brand new discussion forum associated with this group. Please contribute to the discussion.

Spring 2008

This semester the meetings will be on Wednesday from 3:30 to 5:00 in EE/CS 6-212.

April 2 Steve Damer will talk about his work on achieving cooperation between agents in a minimally constrained environment.
March 26 Ham Ching Lam will talk about his work on "Probability Analysis and Modeling of Influenza Type A virus Hemagglutinin Gene with Markov Model."
March 12 - Roman Briskine will talk about his MS project on "Comparison of parameter selection methods for phosphorylation site meta-predictor".
J. Wan et al. recently published a paper that described a weighted voting meta-predictor for phosphorylation site detection. The weights are applied to the predictions returned by a number of base classifiers. To find the weights, J.Wan et al. used restricted grid search. The resulting meta-predictor had better accuracy than any of the base classifiers. However, restricted grid search is a very slow algorithm that checks only a small number of possibilities. Our goal was to find a faster method that produced better results.
February 27 - David Olsen will talk on "An Introduction to Spectral Clustering and Its Place in the Constellation of Clustering Algorithms".
February 20 - Steve Jensen will talk on "Polychronic Neural Nets", which are networks that include the synchronization of inputs/outputs in time. The talk will include a little biological background on neurons and their temporal dynamics, a little ELPH background and how this stuff might tie-in to the ELPH paradigm, and some fairly recent stuff on learning with these sorts of networks.
February 13 - Brett Borghetti on "Measuring Accuracy of Opponent Models: a Call for Action"
February 6 - Wolf Ketter on an evolutionary model for agents.
January 30 - Dan Boley will talk on metabolic pathways.

Fall 2007

This semester the meetings will be on Wednesday from 3:00 to 5:00 in EE/CS 5-212.

December 5 - Mohamed Elidrisi, Muhammad Ahmad, and James Faghmous will talk about their work on poker.
November 28 - Dimitriye Yevremovich will talk on "Computation of Metabolic Pathways".
This talk deals with the computation of the elementary pathways through a metabolic network like those found in E. coli bacteria.
November 21 - No meeting. Happy Thanksgiving!
November 14 - No meeting
November 7 - In preparation for Mike Jordan visit, Hanhuai Shan and Rudi Agovic will lead the discussion on the following papers:
1. Dirichlet processes, Chinese restaurant processes and all that
  M. I. Jordan. Tutorial presentation at the NIPS Conference, 2005.
2. D. M. Blei and M. I. Jordan . Variational inference for Dirichlet process mixtures
More relevant paper/slides can be found in Arindam's class page at http://www-users.itlabs.umn.edu/classes/Fall-2007/csci8980-graph/index.php?page=schedule
October 31 - Tim Clifford will present the paper "Jokes and the Logic of the Cognitive Unconscious" by Marvin Minsky (available at http://citeseer.ist.psu.edu/527323.html)
October 24 - Vassilis Christopoulos will talk on "Handling shape and contact location uncertainty in grasping two-dimensional planar objects"
Abstract: This work addresses the problem of selecting contact locations for grasping objects in the presence of shape and contact location uncertainty. Focusing on two-dimensional planar objects and two finger grasps for simplicity, we present a principled approach for selecting contact points by analyzing the risk of force closure failure. The key contribution of this paper is the development of a method that incorporates shape uncertainty into grasp stability analysis. We propose a grasp quality metric that can be used to identify stable contact regions in the face of shape and contact location uncertainty. The proposed method successfully distinguishes grasps that are equivalent without uncertainty, and we illustrate the properties of this technique with simulation experiments in two classes of objects.
October 17 - Steve Damer will talk about "Latent Dirichlet Co-clustering", a graphical model which is used to cluster documents by topic, assigning topics to each paragraph in the document. To know more read http://ieeexplore.ieee.org.floyd.lib.umn.edu/iel5/4053012/4053013/04053080.pdf?tp=&arnumber=4053080&isnumber=4053013
October 10 - Brett Borghetti on "Foliage: Decision Trees of a Different Color".
This talk will cover four methods for generating and using decision trees are not usually presented in current UMN CS Artificial Intelligence-series classes. The first two methods presented will cover very fast methods to generate "classical" decision trees: one for tree generation when the data comes from a stationary process and one for tree generation from non-stationary process data (data generated by a system with "concept drift"). Both of these methods can be used to generate trees in real time as the data arrives, or generate trees from datasets too large to be comprehensively loaded into working memory. The third method we will discuss involves using oblique (non-axis-parallel) cuts through the data space in order to generate efficient trees with fewer nodes. Finally, we will discuss "random decision forests", collections of multiple dissimilar trees that are processed in a weighted-vote-of-experts scheme to classify new data.
While there is no required reading for this week, for those who are interested, the talk covers 4 papers:
- Mining High-Speed Data Streams (Domingos & Hulten): Sixth Int'l Conf on knowledge discovery and data mining (2000) pp 71-80 http://www.cs.washington.edu/homes/pedrod/papers/kdd00.pdf
- Mining Time-Changing Data Streams (Hulten, Spencer, Domingos): Proc. 7th ACM SIGKDD Int'l Conf. on knowledge discovery and data mining (2001), pp 97-106 http://www.cs.washington.edu/homes/pedrod/papers/kdd01b.pdf
- Induction of oblique decision trees (Heath, Kasif, Salzberg): Proc. 13th IJCAI, pp 1002-1007, Chamb'ery, France, 1993 http://citeseer.ist.psu.edu/95282.html
- Random Decision Forests (Ho): Proc. 3rd Int'l conf on document analysis and recognition, Montreal, Canada, Aug 14-18, 1995, pp278-282 http://citeseer.ist.psu.edu/ho95random.html
also more info on basics of Random Forests & extra sources at Wikipedia: http://en.wikipedia.org/wiki/Random_forest
Slides of the presentation/
October 3 - Tim Miller will talk about "Generative Models of Speech Repair", which is the subject of his PhD dissertation.
September 26 - Ham Ching Lam will talk about his research on
Type A Influenza Virus Hemagglutini Protein Analysis with Directed Acyclic Graph
Hemagglutinin (HA) is an antigenic glycoprotein found on the surface of the influenza viruses. The primary goal of this research is to provide insight into the dominant hemagglutinin amino acid sequences. The study of the influenza virus evolution has traditionally relied on phylogenetic techniques, for example, the maximum likelihood approach and traditional parsimony algorithms. We have proposed simply using a pairwise distance function combined with a directed acyclic graph method to study the evolutionary history of the virus.
September 19 - Anu Pakanati will talk about Cellular Automata

Spring 2007

This semester the meetings will be on Thursday from 2:15 to 4:00 in Walter 405. First meeting on January 25 will be in room 404.

May 3 - More discussion on poker
April 26 - Steve Jensen on ELPH
April 19 - Steve Jensen on ELPH
April 12 - No meeting
April 5 - Steve Damer on "A method for cooperation and detecting cooperation in normal form games"
March 29 - Tim Woods will talk about poker.
March 22 - Brett Borghetti will talk about poker. Read his presentation pokerReserchGroup20070322.pdf
March 9 - Anu Pakanati will present an introduction to Artificial Life, loosely talking from slides prepared from Christopher Adami's book. You can read a rather lengthy but interesting article of peripheral interest to the lecture detailing some of the applications of artificial life.
March 1 - Anu Pakanati will introduce Artificial Life. Canceled because of snow storm.
February 22 - meeting canceled because of Ann Winblad (3:00 in room 3-180).
February 15 - Vassilis Christopoulos will speak on "On-Line Non-Invasive Neural Control of Prosthesis: MEG/EEG.
Many different disorders can destroy the neuromuscular channels through which the brain communicates with and controls its external environment. Amyotrophic Lateral Sclerosis (ALS), Brain or spinal cord injury, Cerebral palsy and Multiple Sclerosis are examples of diseases that disrupt the neural pathways that control muscles or disrupt the muscles themselves. They affect 2,000,000 patients only in USA and far more around the world. Some of them may lose all of voluntary mass control, including eye movement, respiration and may be completely =93locked-in=94 to their bodies unable to communicate with the external worlds. Brain Computer Interface (BCI) can be used to provide communication between the patient and its environment. BCI provides communication and control to people with severe motor disabilities. It can use invasive and non-invasive methods for recording brain signals and controlling prosthetic devices. Invasive BCI uses implanted electrodes to extract the brain signal information that conveys the user=92s commands. However, the implanted electrodes in the brain carry inherent risk, since they show viability of some months and they should be replaced by new electrodes. In this work, we present a new non-invasive BCI technique using Electroencephalography (EEG) for recording the brain signals. A human subject copies a pentagon for 5 minutes using an X-Y=AD joystick while EEG signals are being recorded from 64 (or 32) channels. Finally, a linear summation of weighted contributions of the EEG signals is used to predict the movement trajectory.
Related work:
Georgopoulos AP, Kettner RE, Schwartz AB. (1988) Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population. J Neurosci. 8: 2928-2937.
Georgopoulos AP, Langheim FJ, Leuthold AC, Merkle AN. Magnetoencephalographic signals predict movement trajectory in space. Exp Brain Res. 2005 Nov;167(1):132-5. Epub 2005 Oct 29.
Taylor DM, Helms Tillery SI, and Schwartz AB, Direct Cortical Control of 3D Neuroprosthetic Devices, Science 296: 1829-1832 (2002)
February 8 - We will watch the video on "Importance Sampling vs Markov Chain Monte Carlo" by Diaconis, a professor in Statistics at Stanford. The video is available at http://video.google.com/videoplay?docid=-3554812553412465418&q=markov+chain+mont e+carlo&hl=en. Dan Boley will provide comments.
February 1 - Paul Schrater will continue to talk about his NIPS 2006 paper.
January 25 - Paul Schrater will talk about his NIPS 2006 paper entitled Theory and Dynamics of Perceptual Bistability

Fall 2006

This semester the meetings will be on Wednesday from 3:15 to 5:00 in Walter 405. First meeting on September 13.

November 29 - Paul Schrater will talk about his NIPS 2006 paper entitled Theory and Dynamics of Perceptual Bistability CANCELED
November 14 - Ryan McCabe will present material on Recurrent Events and Renewal Theory. Take a look at Recurrent Events-Renewal Theory-Feller.pdf and the notes from Ryan
November 8 - Evangelos Theodorou will continue his presentation on reinforcement learning.
November 1 - Tim Miller will present on maximum entropy models, since there was some interest in that topic in the information theory meeting.
October 25 - Introduction to Reinforcement Learning, presented by Evangelos Theodorou
October 18 - Introduction to Information Theory, presented by Stephen Wu. Thanks for reading the paper indicated in the email.
October 11 - CANCELED -- Introduction to Reinforcement Learning presented by Evangelos Theodorou
October 4 - Steve Damer continued the introduction to Game Theory. Ryan McCabe has posted an essay to his homepage addressing some of the things we talked about. Feel free to continue the discussion in the forum!
September 27 - Introduction to Game Theory presented by Steve Damer. Paper.
September 20 -- Brett Borghetti talked about the winning rock-paper-scissors-playing agent developed by Dan Egnor described at http://www.ofb.net/~egnor/iocaine.html.
September 13 -- discussion on what to cover this semester

Past year meetings

June 29 -- Tim Miller has presented a demo of the new speech recognition system developed in the NLP lab and talk about its architecture.

June 15 -- Arindam presented his ICML paper.

June 1 -- Wolf Ketter did a dry run of his job interview talk.

27 April 2006 - Dongwei Cao talked about his work.
On Approximate Support Vector Machine and its Generalization Performance

We study the algorithms that speed up the training process of support vector machines by using an approximate solution. We focus on algorithms that reduce the size of the optimization problem by extracting from the original training data set a small number of representatives and using these representatives to train an approximate SVM. Using the approach of algorithmic stability, we prove a PAC-style generalization bound for the approximate SVM, which includes as a special case the generalization bound for the exact SVM, which is trained using the original training data set.

20 April 2006 - no meeting

13 April 2006 - no meeting

6 April 2006 - Evangelos Theodorou talked about the newer inverse reinforcement learning by Andrew Ng. Paper.

30 March 2006 - Evangelos Theodorou talked about inverse reinforcement learning. Paper

23 March 2006 - Tim Miller talked about hierarchical Hidden Markov Models (HHMMs), their relation to Dynamic Bayes Nets (DBNs), and how they are used in the NLP lab. Paper - Slides

16 March 2006 - Spring break - no meeting

9 March 2006 - William Schuler talked about DBNs and the speech recognition odyssey of the NLP lab.

2 March 2006 - Steve Damer will be talking about his recent work.

23 February 2006 - .. and now we move to MRF. The paper is http://www.cs.cornell.edu/~rdz/Papers/BVZ-cvpr98.pdf. Paul will lead the discussion.

16 February 2006 - Now that we have learned everything about CRFs in theory, we will look at an application to computer vision. The paper we will look at is Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes, by Murphy, Torralba, and Freeman. Nate Bird will be presenting the paper.

9 February 2006 - We will move forward to Conditional Random Fields (CRFs) by reading the paper Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data by Lafferty, McCallum, and Pereira. Other resources are two tutorials (by (1) Wallach and (2) Sutton and McCallum).

2 February 2006 - We will read and discuss a paper on Maximum Entropy models, and if time permits we will study their relevance to predicting how many more weeks of winter we will have. There is a web tutorial on Adam Berger's webpage, as well as a paper on applying maxent models to NLP which is similar to the tutorial through the first 4 sections.

26 January 2006 - For our first meeting this semester, Dongwei Cao will be talking about the work he will be presenting at the SIAM data mining conference, "On Approximate Solutions to Support Vector Machines."

8 December 2005 - This week we will be looking at the paper "Factor Graphs and the Sum-Product Algorithm" by Kschischang, Frey, and Loeliger. Link

For a brief outline, see this site: Link

1 December 2005 - We decided to do a review on Hidden Markov Models (HMMs). We covered them last year, but this will help get new people up to speed and the old people to clear out the cognitive cobwebs. This is all in anticipation of learning more about undirected graphical models like Markov Random Fields, and then possibly moving to Conditional Random Fields, and after that we should know everything about machine learning. Link to Rabiner paper

24 November 2005 - No meeting - Thanksgiving holiday. Remember to give thanks for the outobox webmaster.

17 November 2005 - Continuation of last week's topic of graphical models.

10 November 2005 - Towards the end of last weeks session we started talking about the application of Markov Random Fields to Miguel's problem of detecting proteins from electron microscopy images. We had so much fun we decided to look a little bit at MRFs and graphical models in general. So, for next week, we will be reading a tutorial by Kevin Murphy which you can find at this location. There is also a tech report by Wainwright and Jordan at this location (postscript format) which is quite long, but fear not! The intro is a nice overview of applications of graphical models and is quite readable I'm told.

3 November 2005 - Miguel, a visiting graduate student from Spain, will talk about his work done there which he is continuing here.

27 October 2005 - We will again not be having a meeting, in observation of Theodore Roosevelt's birthday. If you would like something to mull over the two-week break, think about how you would build an agent to wash dishes by hand. In order to get a feel for the scope of the problem, you are welcome to visit my apartment and try the task out for yourself.

20 October 2005 - We looked at a paper by Friedman, Hastie, and Tibshirani - "Additive Logistic Regression: A statistical view of boosting." Citeseer link is here. The diligent reader may also be interested in this paper "Why the logistic function? A tutorial discussion on probabilities and neural networks" by Michael I. Jordan.

13 October 2005 - No meeting because of Navy Day Holiday

6 October 2005 - We read an introduction paper on boosting, which helped to motivate and clarify the paper we read last week. Here is a link to a pdf version of the paper "A short introduction to boosting," and here is a link to the original adaboost paper, "A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting."

29 September 2005 - Discussed the paper "Logistic Regression, Adaboost, and Bregman Distances" by Collins et al. (2001). Citeseer link

22 September 2005 - Harini Veeraraghavan talked about her research regarding event detection in videos of automobile traffic.

15 September 2005 - New faculty member Arindam Banerjee talked about his research on Bregman Divergences.
Slides

2004-2005

Some people have asked for a list of papers the group looked at last year. If anyone has any additions let me know.

Bayesian Networks without Tears - Eugene Charniak
Learning Dynamic Bayesian Networks - Zoubin Ghahramani
A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition - Lawrence R. Rabiner