Machine Learning (2017 Fall)

Course staff

Insturctor: Chenhao Tan contact
Office hours: 4:30-5:30pm Thursday and by appointment (ECES 118A)

Graduate assistants:

• Zhenguo Chen contact
  Office hours: 11am-12pm on Monday and Wednesday at ECCR 1B50B
• Tyler Scott contact
  Office hours: 2-3pm on Tuesday and 11am-12pm on Thursday at ECCR 1B50F

Graders:

• Zhenguo Chen contact
• Sean Harrison contact

Logistics

• Location and time: ECCS 1B12, 5:00-6:15pm on Mondays and Wednesdays
• Syllabus (Must READ if you are taking the course)
• Final project description
• Webpage: https://chenhaot.com/courses/csci5622/2017fa/home.html
• Piazza link: https://piazza.com/class/j4htdllqf3a6n0
• Textbooks (The lectures should be self-contained, but these books should help you, especially if you would like to dive deeper.)
  • Machine Learning: A Probabilistic Perspective (Murphy)
  • Understanding Machine Learning: From Theory to Algorithms (SSBD)
  • Elements of Statistical Learning (HTF)
  • Foundations of Machine Learning (MRT)

Prerequisites

Mathematical background: Please finish these questions and submit by the second week of the semester. This does not contribute to your final scores, but it helps both yourself and the instructors to know your mathematical background.

Programming: We will make extensive use of the Python programming language. Most homeworks will be in python3.

Course evaluation

• homeworks (5 times, each 10% and the lowest score will be excluded in the final score, 40%)
• midterm (15%)
• final project (40%, the assignment is out starting from the first day, start early on this!)
• participation (5%)

Schedule

• Week 1
  • Aug 28, Introduction
  • Aug 30, K-nearest neighbor
    Readings: SSBD chapter 1, (Assumed background) SSBD Appendix C, (Assumed background) Joint Probabilities and MLE
    More background reading (from Stanford CS224n): Linear algebra, Probability, Convex optimization
    HW1 out on Sep 1
• Week 2
  • Sep 4, Labor day
  • Sep 6, Bias-variance tradeoff and model evaluation
    Readings: HTF chapter 7
    Leakage in data mining: Formulation, detection, and avoidance
• Week 3
  • Sep 11, Naive Bayes and logistic regression
  • Sep 13, Gradient descent and stochastic gradient descent
    Readings: Mitchell Machine learning Ch. 3
    Notes on logistic regression
    On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes by Andrew Y. Ng and Michael I. Jordan
    Lazy sparse stochastic gradient updates by William Cohen
    An overview of gradient descent optimization algorithms
    HW1 due, HW2 out on Sep 15
• Week 4
  • Sep 18, Neural networks and deep learning
  • Sep 20, Backpropagation
    Readings: SSBD chapter 20
    Convolutional neural networks
    Recurrent neural networks (LSTM)
    Backprop is not just the chain rule
• Week 5
  • Sep 25, Common techniques in deep learning
  • Sep 27, Max-margin learning, SVM (Annotated on Oct 2)
    Readings: Keras Tutorial, must read for homework
    Stanford CS231n materials: Setting up the data and the loss, Learning and Evaluation
    Why Momentum Works
    SVM Tutorial
    MRT chapter 4
    HW2 due, HW3 out
• Week 6
  • Oct 2, Duality (Annotated on Oct 2, Annotated on Oct 4)
  • Oct 4, Kernels(Annotated on Oct 4)
    Readings: SSBD chapter 15 & 16
    Final project group formation due
• Week 7
  • Oct 9, Regularization, regression and multi-class classification
  • Oct 11, Ensembling/boosting
    Readings: SSBD chapter 10 & 17
    HTF chapter 3
    HW3 due, HW4 out
• Week 8
  • Oct 16, Learning theory I
  • Oct 18, Final project pitches
    Readings: SSBD chapter 2 & 3
    Final project proposal due on Oct 17
• Week 9
  • Oct 23, Learning theory II
  • Oct 25, Dimensionality reduction, PCA/SVD
    Readings: SSBD chapter 23
    Final project proposal peer feedback due
• Week 10
  • Oct 30, Review
  • Nov 01, Midterm
• Week 11
  • Nov 06, Midterm recap
  • Nov 08, Clustering
    Readings: SSBD chapter 22
    HW4 due, HW5 out
• Week 12
  • Nov 13, Expectation-minimization algorithms, Topic models
  • Nov 15, Final project midpoint presentation
    Readings: SSBD chapter 24.4, Chapter 1 of Applications of Topic Models
    Final project midpoint slides due on Nov 14
• Week 13 (Fall break & Thanksgiving!)
• Week 14
  • Nov 27, Final project office hour
  • Nov 29, Variational inference
    Readings: Eisner, Blei, Latent Dirichlet Allocation
    Final project midpoint peer feedback due on Nov 27
• Week 15
  • Dec 4, Reinforcement Learning I
  • Dec 6, Reinforcement learning II
    Readings: RL Book chapter 1, 3, 4, 5, 6
    HW5 due
• Week 16
  • Dec 11, Ethics, Machine learning, and society
    Readings: NIPS 2017 invited talk by Kate Crawford
  • Dec 13, Final project presentation (DLC 1B70 10:00AM-12:00PM, make your poster shine! Refreshments will be provided.)
    Final project report due on Dec 15