CMSC 422

Introduction to Machine Learning

Fall 2019

Syllabus

 

Sections

This class is offered in two independent sections. The homework and final exam will be the same for both classes, though the mid-term exams will be different.  Students in either section should feel free to attend office hours with any of the instructors or TAs.

Section	Instructor	Time	Classroom
0101	David Jacobs	MW 2:00 pm - 3:15 pm	CSI 1115
0201	Ramani Duraiswami	TuTh 9:30 am - 10:45 am	IRB 0318

 

 

Personnel
	Instructor	Instructor	Teaching Assistants
Name	David Jacobs	Ramani Duraiswami	Juan Luque	Justin Shen	Alexander Hoyle	Yow-Ting Shiue	Chen Chen	Runxing Lin
Office	IRB 4240	IRB 4244	IRB 1108	IRB 1108	IRB 1108	IRB 1108	IRB 1108	IRB 1108
Office Hours	Mon 9-10, Tues 11-12	Wed 10-12	M 3:30-5:30	Tu 1-3	Fri 9-11	W 9-11	Th 1-3	Tues 9-11
Email	djacobs (at) cs	ramani (at) cs

 

 

Prerequisites

CMSC320, CMSC330, and CMSC351, one of (MATH240, MATH461); and permission of CMNS-Computer Science department.

There will be a lot of math in this class and if you do not come prepared, life will be rough.

·      You must be able to take derivatives by hand (preferably of multivariate functions).

·      You must know what the chain rule of probability is, and Bayes' rule.

·      Linear algebra would be helpful, but is not necessary: for instance, dot products and their relationship to projections onto subspaces.

·      If you haven't at least seen these things before, you will need to invest a significant amount of time to catch up on math background.

On the programming side, projects will be in Python; you should understand basic computer science concepts (like recursion), basic data structures (trees, graphs), and basic algorithms (search, sorting, etc.).

 

Grading

The purpose of assignments & grading is to provide extra incentive to help you keep up with the material and assess how well you understand it, so that you have a solid background in machine learning by the end of the semester.

We expect students to

- Come to class prepared, having completed the assigned readings.

- Complete the assigned homework assignments before the class its due, and be prepared to discuss their solution in class.

 - Participate actively in discussions both in person and online.

Your grade will be based on:

• Homeworks and Implementations (35%). There will be nine of these. Some of these will be regular problem sets, which are due the following week, while some will be implementations of machine learning algorithms in Python, for which you have two weeks. These are to be completed individually.
• Midterm exam (25%). There will be an in-class midterm exam, obviously to be completed individually. You may bring one handwritten sheet of notes, prepared by you.
• Final exam (40%). There will be a (cumulative) final exam, during the official slot, to be completed individually. You may bring one handwritten sheet of notes, prepared by you.

Communication and Discussion

Both sections will be using Piazza for class-related discussion and communication. The system is highly catered to getting you help fast and efficiently from classmates, the TAs, and the instructors. Rather than emailing questions to the teaching staff, please post your questions on Piazza (either as public discussions or as private posts to instructors). Piazza allows both anonymous and public messages. All messages sent to the instructors email addresses will be initially redirected to Piazza, and later ignored.

Find our class page at: https://piazza.com/umd/fall2019/cmsc422/home

 If you have any problems or feedback for the developers, email team@piazza.com.

 

Policies

Lateness

Late homeworks are not allowed. Period. No exceptions. The time deadlines are automatic and unforgiving. If you handed something in and do not get a score for an assignment, you have one week to let us know about the issue.

Excused Absences

Any student who needs to be excused for an absence from a single lecture, recitation, or lab due to a medically necessitated absence shall:

• a) Make a reasonable attempt to inform the instructor of his/her illness prior to the class via piazza.
• b) Upon returning to the class, present their instructor with a self-signed note attesting to the date of their illness. Each note must contain an acknowledgment by the student that the information provided is true and correct. Providing false information to University officials is prohibited under Part 9(i) of the Code of Student Conduct (V-1.00(B) University of Maryland Code of Student Conduct) and may result in disciplinary action.

The self-documentation may not be used for the Major Scheduled Grading Events as defined below and it may only be used for only 1 class meeting (or more, if you choose) during the semester. Any student who needs to be excused for a prolonged absence (2 or more consecutive class meetings), or for a Major Scheduled Grading Event, must provide written documentation of the illness from the Health Center or from an outside health care provider. This documentation must verify dates of treatment and indicate the timeframe that the student was unable to meet academic responsibilities. In addition, it must contain the name and phone number of the medical service provider to be used if verification is needed. No diagnostic information will ever be requested. The Major Scheduled Grading Events for this course include: the midterm exam, and the final exam.

 

Disability Support

Any student eligible for and requesting reasonable academic accommodations due to a disability is requested to provide, to the instructor in office hours, a letter of accommodation from the Office of Disability Support Services (DSS) within the first TWO weeks of the semester.

 

Academic Integrity

In this course you are responsible for both the University’s Code of Academic Integrity and the University of Maryland Guidelines for Acceptable Use of Computing Resources. Any evidence of unacceptable use of computer accounts or unauthorized cooperation on tests and assignments will be submitted to the Student Honor Council, which could result in an XF for the course, suspension, or expulsion from the University.

Note that posting project solutions in a public online location is a violation of your academic integrity policy.

Any homework or exam that is handed in must be your own work. However, talking with one another to understand the material better is strongly encouraged. Recognizing the distinction between cheating and cooperation is very important. If you copy someone else's solution, you are cheating. If you let someone else copy your solution, you are cheating. If someone dictates a solution to you, you are cheating. Everything you hand in must be in your own words, and based on your own understanding of the solution. If someone helps you understand the problem during a high-level discussion, you are not cheating. We strongly encourage students to help one another understand the material presented in class, in the book, and general issues relevant to the assignments. When taking an exam, you must work independently. Any collaboration during an exam will be considered cheating. Any student who is caught cheating will be given an E in the course and referred to the University Student Behavior Committee. Please don't take that chance - if you're having trouble understanding the material, please let us know and we will be more than happy to help.

 

Anti-Harassment

The open exchange of ideas, the freedom of thought and expression, and respectful scientific debate are central to the aims and goals of this course. These require a community and an environment that recognizes the inherent worth of every person and group, that fosters dignity, understanding, and mutual respect, and that embraces diversity. Harassment and hostile behavior are unwelcome in any part of this course. This includes: speech or behavior that intimidates, creates discomfort, or interferes with a person’s participation or opportunity for participation in the course. We aim for this course to be an environment where harassment in any form does not happen, including but not limited to: harassment based on race, gender, religion, age, color, national origin, ancestry, disability, sexual orientation, or gender identity. Harassment includes degrading verbal comments, deliberate intimidation, stalking, harassing photography or recording, inappropriate physical contact, and unwelcome sexual attention. Please contact an instructor or CS staff member if you have questions or if you feel you are the victim of harassment (or otherwise witness harassment of others).

 

Course evaluations

We welcome your suggestions for improving this class, please don’t hesitate to share it with the instructor or the TA during the semester! You will also be asked to give feedback using the CourseEvalUM system at the end of the semester. Your feedback will help us make the course better.

 

Right to change information

Although every effort has been made to be complete and accurate, unforeseen circumstances arising during the semester could require the adjustment of any material given here. Consequently, given due notice to students, the instructor reserves the right to change any information on this syllabus or in other course materials.

 

Textbook

Our primary source of readings will be A Course in Machine Learning (CIML), a collection of notes by Prof.  Hal Daumé III, which provides a gentle and thorough introduction to the field of machine learning. We will use another online book by Sutton and Barto (S&B) for reinforcement learning.

Other recommended (but not required) books:

• Machine Learning: The Art and Science of Algorithms that Make Sense of Data by Peter Flach (ISBN 1107422221)
• Pattern Recognition and Machine Learning by Chris Bishop (ISBN 0387310738)
• Machine Learning by Tom Mitchell (ISBN 0070428077)
• Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman (ISBN 0387952845)
• Information Theory, Inference and Learning Algorithms by David MacKay (ISBN 0521642981)
• An Introduction to Computational Learning Theory by Michael Kearns and Umesh Vazirani (ISBN 0262111934)

Below is a schedule for problem sets and lectures.  There may be some changes in these as the semester progresses.  PS shown in red indicates schedule for Prof. Duraiswami, PS shown in black shows schedule for Prof. Jacobs.

Problem Set Schedule
	Duraiswami		Jacobs		Topic
	Assigned	Due	Assigned	Due
PS1	Thurs. 8/29	Thurs. 9/5	Wed. 8/28	Wed. 9/4	Review
PS2	Thurs. 9/5	Thurs. 9/12	Wed. 9/11	Wed. 9/18	Perceptrons
PS3	Wed. 9/18	Wed. 9/25	Wed. 9/18	Wed. 9/25	Gradient Descent
PS4	Wed. 9/25	Wed. 10/9	Wed. 9/25	Wed. 10/9	Implementation of Neural Networks
PS5	Wed. 10/9	Wed. 10/16	Wed. 10/9	Wed. 10/16	Probabilistic Models
Midterm	Thurs. 10/17		Mon. 10/21
PS6	Mon. 10/21	Mon. 11/4	Mon. 10/21	Mon. 11/4	Implementation of E-M
PS7	Mon. 11/4	Mon. 11/18	Mon. 11/4	Mon. 11/18	Implementation of logistic regression, PCA
PS8	Mon.  11/18	Mon. 11/25	Mon. 11/18	Mon. 11/25	SVMs
PS9	Mon. 12/2	Mon. 12/9	Mon. 12/2	Mon. 12/9	Learning Theory, Bias and Fairness

 

 

Lec	Date	Content	Source	Problem Set
1.	08/26 08/27	Introduction -- including discussion of what we expect you to already know.	CIML 1, 2, 3, 15-15.2. Math for Machine Learning
2.	08/28 08/29	Review material on regression, classification, clustering, data sets, distributions, overfitting/generalization		PS1 on review material PS1
3.	09/04 09/03	Linear classifiers: Perceptrons	CIML 4-4.5
4.	09/09 09/05	Perceptron continued	CIML 4.5-4.7	PS2
5.	09/11 09/10	Practicalities of ML, including overfitting	CIML 5-5.6	PS2 on Perceptrons
6.	09/16 09/12	Linear Classifiers and Gradient Descent	CIML 7-7.4	PS3
7.	09/18 09/17	Neural Networks, how a trained network works.	CIML 7.5, 10-10.1	PS3 on Gradient Descent
8.	09/23 09/19	Neural Networks, training the network.	CIML 10.2-10.3	PS4
9.	09/25 09/24	Deep Learning	CIML 10.6	PS4 Implementation of NNs
10.	09/30 09/26	Deep Learning cont’d	TBD
11.	10/02 10/01	Probabilistic Models	CIML 9-9.5
12.	10/07 10/03	Probabilistic Models cont’d	CIML 9.6-9.7	PS5
13.	10/09 10/08	Mixture of Gaussians	TBD	PS5 on Probabilistic Models
14.	10/14 10/10	E-M	TBD
15.	10/16 10/15			PS 5 Due Midterm practice problems
16.	10/21 10/17	Midterm		PS6 PS6 Implementation of E-M
17.	10/23 10/22	Go over midterm
18.	10/28 10/24	Linear regression, logistic regression	CIML 7.6, 9.6
19.	10/30 10/29	Principal Component Analysis (PCA)	CIML 15.2
20.	11/04 10/31	Kernels	CIML 11-11.3
21.	11/06 11/05	SVM	CIML 11.4-11.6
22.	11/11 11/07	SVM cont’d
23.	11/13 11/12	Learning Theory	CIML 12
24.	11/18 11/14	Reinforcement learning
25.	11/20 11/19	Deep Reinforcement Learning
26.	11/25 11/21	Generative Adversarial Networks (GANs)	TBD	PS8 Due PS8 Due Practice problems for final.
27.	12/02 11/26	Bias and Fairness	CIML 8	PS9 PS9 Learning Theory, Bias and Fairness.
28.	12/04 12/03	Review, practice solving problems
29.	12/09 12/05	Conclusions, AI and Society		PS9 Due PS9 Due
30	12/16	Common Final: 6:30-8:30pm