Special Topics in Computer Science:

 

Machine Learning

CIS 595 – Spring 2002

 

Course materials, Homework...

 

Meeting days:

Monday, 7:25P - 9:55P, TL302

 

Instructor:

Slobodan Vucetic, 323 Wachman Hall, phone: 204-5773, www.ist.temple.edu/~vucetic

 

Office Hours:

Wednesday 2:00 pm - 3:00 pm, or by appointment

 

Objective:

 

The goal of the field of machine learning is to build computer systems that learn from experience and that are capable to adapt to their environments. Learning techniques and methods developed by researchers in this field have been successfully applied to a variety of learning tasks in a broad range of areas, including, for example, text classification, gene discovery, financial forecasting, credit card fraud detection, collaborative filtering, design of adaptive web agents and others.

 

This introductory machine learning course will give an overview of many techniques and algorithms in machine learning, beginning with topics such as simple concept learning and ending up with more recent topics such as boosting, support vector machines, and reinforcement learning. The objective of the course is not only to present the modern machine learning methods but also to give the basic intuitions behind the methods as well as a more formal understanding of how and why they work.

Required texts:

· T.M. Mitchell, Machine Learning, 1997.

· R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2000.

Additional papers and handouts relevant to presented topics will be distributed as needed.
 

Topics:

· Concept learning. Version spaces. PAC learning. VC dimension.

· Regression. Loss function. Least-squares fit. Parameter estimation. Statistical view on the regression. Log likelihood measures. On-line learning techniques.

· Classification. Logistic regression. Class-conditional densities. Parameter estimation. Perceptron algorithm. On-line techniques. Multiple classes.

· Neural networks. Nonlinear decision boundaries. Backpropagation. Radial basis functions.

· Classification and regression trees CART. C4.5.

· Support vector machines. Classification. Max margin hyperplanes. Kernel functions.

· Ensemble methods. Mixture of experts. Bagging. Boosting.

· Unsurpervised learning. Clustering, k-means.

· Density estimation. Parametric methods. Mixture of Gaussians. Non-parametric. Parzen windows. Nearest neighbor.

· Dimensionality reduction. Feature extraction. Mutual information measure. PCA. Clustering.

· Bayesian networks. Independence structure. Inference. Parameter learning. Structure learning.

· Learning with hidden variables and missing data. Expectation-maximization algorithm.

· Hidden Markov models. Forward, backward algorithm. Baum-Welch algorithm.

· Markov random fields. Independence structure. Inference. Learning.

· Reinforcement learning. Learning to act. Markov decision processes. Reinforcement learning with delayed rewards.
 

Grading:

A combination of homework assignments (40%), a midterm (20%), and an individual project (40%)