Introduction to Machine Learning

This course will introduce the basic principles of Machine Learning.  These include simple but useful machine learning algorithms and the mathematics underlying these methods.  Recently, Large Language Models have had a large and growing impact on our lives.  In this class, we will discuss some of the fundamental ideas that underlie these models, providing a basic understanding of how they operate.  We will also look at some of the implications to society of large scale machine learning.

Assignments

Students will be assigned seven problem sets and also receive a number of in-class assessments.

Problem Sets: The schedule for problem sets is given below.  These will involve implementing algorithms and pencil and paper exercises.  Students may make use of LLMs for assistance and may work with other students.  Students are expected to work through each assignment, and to understand all pieces of code and all written solutions that they turn in.  Students will have two weeks to complete coding assignments (which may also contain written problems), and one week for assignments that are purely written.  All problem sets are due at start of class (2pm) on the due date. Two week assignments will count for twice as much as one week assignments in grading.  Problem sets will count overall for 16½ % of the final grade.

Quizzes: There will be seven short (ten minute) quizzes given in class.  These quizzes will be given on the date that each problem set is due, and will address the material in that problem set.  There will be no partial credit given on quizzes.  These will count for 16 1/2% of the final grade.

Midterm and Final: An in-class midterm will be given (see schedule below) and a final exam will be given at the date, time and location set by the university schedule (see schedule below).  The midterm will could for 27% of the final grade and the final exam will count for 40%.

Some portions of assignments and exams may be listed as extra credit.  This means that all grades will be assigned as usual, ignoring extra credit.  After grades are determined, extra credit work may be used to raise students’ grades.

Texts and Discussion

Readings will be given in the schedule below.  We will use the following texts, along with some additional pointers.

CIML - A Course in Machine Learning, by Hal Daume.  It is available freely here.  Or here.

ISL – An Introduction to Statistical Learning, by James, et al.  It is freely available here.

Neural Networks and Machine Learning, by Michael Nielsen.  It is freely available here.

We will be using Piazza for discussion: https://piazza.com/umd/spring2025/cmsc4220201

Disability Support

Any student eligible for and requesting reasonable academic accommodations due to a disability is requested to provide, to the instructor in office hours, a letter of accommodation from the Office of Disability Support Services (DSS) within the first TWO weeks of the semester.

Academic Integrity

In this course you are responsible for both the University’s Code of Academic Integrity and the University of Maryland Guidelines for Acceptable Use of Computing Resources. Any evidence of unacceptable use of computer accounts or unauthorized cooperation on tests, quizzes and assignments will be submitted to the Student Honor Council, which could result in an XF for the course, suspension, or expulsion from the University.

Any work that you hand in must be your own work.  Any sources that you draw from, including other students or LLMs, should be appropriately acknowledged.  Plagiarism is a serious offense, and will not be tolerated.  This should not be an issue with assignments; you just need to acknowledge any sources that you use.

Anti-Harassment

The open exchange of ideas, the freedom of thought and expression, and respectful scientific debate are central to the aims and goals of this course. These require a community and an environment that recognizes the inherent worth of every person and group, that fosters dignity, understanding, and mutual respect, and that embraces diversity. Harassment and hostile behavior are unwelcome in any part of this course. This includes: speech or behavior that intimidates, creates discomfort, or interferes with a person’s participation or opportunity for participation in the course. We aim for this course to be an environment where harassment in any form does not happen, including but not limited to: harassment based on race, gender, religion, age, color, national origin, ancestry, disability, sexual orientation, or gender identity. Harassment includes degrading verbal comments, deliberate intimidation, stalking, harassing photography or recording, inappropriate physical contact, and unwelcome sexual attention. Please contact an instructor or CS staff member if you have questions or if you feel you are the victim of harassment (or otherwise witness harassment of others).

Course evaluations

We welcome your suggestions for improving this class, please don’t hesitate to share it with the instructor or the TAs during the semester! You will also be asked to give feedback using the CourseEvalUM system at the end of the semester. Your feedback will help us make the course better.

Web Accessibility

Personnel

Instructor

Teaching Assistants

Name

David Jacobs

Jiaye Wu

Bhuvanesh Murali

Haowen Yu

Naitri Rajyaguru

Office

IRB 4240

Email

djacobs@cs

Please post on Piazza

Office Hours

Friday 9-11, IRB 4240

Monday 3:15-5:15

AVW 4140

Tuesday, 2-4

Wednesday, 10-12

Thursday, 9:30-11:30

Homework

Date Assigned

Date Due

Topic

PS1

Feb. 3

Feb. 10

Decision Trees

PS2

Feb. 12

Feb. 24

K-means - Implementation

PS3

Feb. 24

March 3

Perceptrons, Naïve Bayes

PS4

March 3

March 24

Logistic Regression - Implementation

PS5

March 31

April 14

Neural Networks, Backpropagation, Gradient Descent - Implementation

PS6

April 14

April 28

Reinforcement Learning - Implementation

PS7

April 28

May 5

SVM, Learning Theory

Topic

Assigned Reading

Additional resources

1.  1/27

Introduction

2. 1/29

Decision Trees, Limits of Learning

CIML, Chapter 1, 2

3. 2/3

Decision Trees cont’d, ensemble learning.

4. 2/5

Geometry Review

Dot product notes

TBP (To be Posted)

5. 2/10

Nearest Neighbor (NN), K-NN, K-Means -Quiz

CIML, Chapter 3

6. 2/12

Snow Day

7. 2/17

Perceptrons, multi-variable calculus, gradient descent.

CIML, 4-4.4

8. 2/19

Perceptrons, cont’d.

CIML, Chapter 5

9. 2/24

Practicalities of Machine Learning - Quiz

Notes

10. 2/26

Probability  & Probabilistic Models

CIML 9-9.5

11. 3/3

Linear and Logistic Regression - Quiz

ISL 3.1, 4.3

Part II of Notes

12. 3/5

Logistic Regression, cont’d.

13. 3/10

Neural Networks

CIML 10.1, Neural Networks and Deep Learning Chapter 1

14. 3/12

Backpropagation

CIML 10.2, Neural Networks and Deep Learning Chapter 2.

Spring Break

15. 3/24

Next Word Prediction - Quiz

Prediction and Entropy of Printed English, by Shannon, Bell System Technical Journal.

16. 3/26

Transformers

Attention is all you need,  Vaswani et al., Neurips 2017.

Formal Algorithms for Transformers by Phuong and Hutter, Arxiv.  I highly recommend this for a clear and precise description of Transformers, but it is not for the faint of heart.

17. 3/31

Midterm

.

18. 4/2

Going over Midterm

.

19. 4/7

Reinforcement Learning

Reinforcement Learning, an Introduction, Sutton and Barto Understanding Chapters 3 and 6 is important, but reading 4 and 5 will probably help with 6.  Chapter 1 is fun and quick to read.

20. 4/9

Large Language Models

Training language models to follow instructions with human feedback

LLaMA: Open and Efficient Foundation Language Models

Language Models are Few-Shot Learners

21. 4/14

Data - Quiz

TBP

22. 4/16

Learning Theory

CIML 12

23. 4/21

Kernel Methods

CIML 11-11.3

24. 4/23

Support Vector Machines

CIML 11.4-11.6

25. 4/28

Kernel SVMs - Quiz

26. 4/30

Bias and Fairness

TBP

27. 5/5

TBD, Maybe Image/Video Generation, PCA, EM. - Quiz

28. 5/7

TBD

29. 5/12

Conclusions

5/19 4-6pm

Final Exam