CISC 5800: Machine Learning

CISC 5800: Machine Learning

Class times: Wednesday, 5:30 – 7:45pm, LL 307
Instructor: Prof. Daniel D. Leeds (my homepage)
Office: LL 610H for Office Hours; normally JMH 332 (Rose Hill)
E-mail:
Office hours: Wednesday 4-5pm

Full syllabus is here.

Course text: No book will be required, but the following will be useful as references.

"Pattern Recognition and Machine Learning" by Bishop
"Machine Learning: A Probabilistic Approach" by Murphy

Programming: We will have programming assignments throughout the semester. I will require you complete your programming assignments in Matlab or Python. There are several ways to use Matlab or Python:

You can run Matlab in the computer lab or (without graphics) by remote login to our department machines.
You can download your own student license for $50-$100.
If you have a strong computational background/past Python experience and want to use Python, you can use numpy, scipy and matplotlib packages to write acceptable code for this class in Python. (You also may used the pandas package; you may not use packages like tensorflow for this class.)

Sections below:

Resources
Announcements
Slides
Assignments

Resources:
Computing guides
Linux Commands - important Linux commands for working on erdos
vi Commands - important commands for the vi text editor; you are welcome to use emacs instead of vi
A Guide to Putty - Information for Windows users on accessing erdos
Note, to access erdos on Mac: (1) Open terminal, (2) type "ssh username@erdos.dsm.fordham.edu" , inserting your username before the @
Matlab
Extra background on Matlab
Matlab programming practice — download the accompanying data file sampleData.mat and function file newFunc.m
Calculus
Calculus practice - some optional problems with derivatives
Midterm practice
Practice questions
Practice questions answers -- AIC answers corrected March 5, 11:40pm! Second derivative answer corrected March 6, 2:40pm
Practice code questions
Practice code questions answers
Final practice
Practice questions -- additional questions two lines below; NOTE Forwards-Backwards actually will NOT be covered on final
Practice answers
Additional practice, coded in colors red, green, blue
Additional practice answers BLUE
Additional practice answers GREEN
Additional practice answers RED

Announcements:
Apr 19 11:4apm: Wednesday office hours next week are cancelled. However, I will be at LC (LL 610H) for office hours on Monday 4-5pm.
The final exam will be May 9 (postponed from May 2).
Mar 11 7:30pm: Wednesday office hours this week are cancelled. However, I will be at LC (LL 610H) for office hours this Monday 4-5pm.
Mar 7 1:50pm: As announced by e-mail, the midterm is postponed until NEXT Wednesday, due to snow
Feb 23 1:15pm: Midterm practice questions posted above!
Feb 20 4:10pm: As announced in class last week, the midterm will be March 7.
Jan 19 3:20pm: Office hours the week of Jan 22 and Jan 29 will be on Monday 4-5pm (LL 610H) not on Wednesday.

Slides: Check slides after lectures as well for updates made during class!
I have not found any one textbook or online resource to be an optimal match for the material we cover at the mathematical level we cover it. However, Andrew Ng's lecture notes, available on Stanford's Machine Learning course web site, often can be a helpful online read. I recommend these notes as well as looking through one of our course textbooks.

Supplementary reading

Lecture 1, Course logistics, background math, intro to classifiers.

Lecture 1.5, Matlab intro.

Lecture 2, Bayes classifier. Ng notes 2 particularly pages 8-11

Lecture 3, Logistic classifier. Ng Notes 1 Elements of Part II (starting on page 16)

Lecture 4, Support vector machines. Ng notes 3 parts of pages 1-20

Lecture 5, Dimensionality reduction. Ng notes 10 on PCA and Ng notes 11 on ICA

Lecture 6, Neural networks. Chapter 1 and Chapter 2 of "Neural Networks and Deep Learning

Lecture 7, Bayes Networks. Murphy (UBC) notes, first few pages are most relevant

Lecture 8, Hidden Markov Models. Stanford notes

Lecture X, Convolutional Neural Networks.

Lecture 9, Learning Theory.

Assignments:
Homework 0 due Jan 24 - several clarification updates made in red on Jan 22, 4:30pm

Homework 0 answers now available

90-101 A range

80-90 B range

70-80 C range

Homework 1 due Feb 7 (written), Feb 9 (code) - data available here as hw1data.mat
Part A questions 2 and 3 were more tricky than intended. I have now provided revised versions in red. You may answer either the original versions (in black) or the new versions (in red) for these two questions.

Homework 1 answers now availabe; grade assignment (based on full answers)

88-98 "A range"

78-88 "B range"

68-78 "C range"

Homework 2 due Feb 21 (written), Feb 23 (code) - data available here as hw2data.mat

Homework 2 answers now availabe (code now available too - Mar 7, 11:45pm); full grade assignment

84-96 "A range"

67-84 "B range"

54-67 "C range"

Midterm Answers

86-106 "A range"

61-86 "B range"

41-61 "C range"

Final project - due May 9 (you can submit until May 13)

Homework 3 due Apr 11 (written, neural net updates made Mar 24), Apr 13 (code) - code part now posted; data file available as emnist_letters.mat q4 example code corrected Mar 25

Homework 3 WRITTEN answers, full grades

110-130 "A range"

80-110 "B range"

50-80 "C range"

Homework 4 due Apr 25 (only written), optional; if you do well, it will be incorporated into your final homework grade; if you do poorly, I will ignore this homework for your final grade

Homework 4 answers
Correction made in Part A, 3b on April 23, 10:30am

	Supplementary reading
Lecture 1, Course logistics, background math, intro to classifiers.
Lecture 1.5, Matlab intro.
Lecture 2, Bayes classifier.	Ng notes 2 particularly pages 8-11
Lecture 3, Logistic classifier.	Ng Notes 1 Elements of Part II (starting on page 16)
Lecture 4, Support vector machines.	Ng notes 3 parts of pages 1-20
Lecture 5, Dimensionality reduction.	Ng notes 10 on PCA and Ng notes 11 on ICA
Lecture 6, Neural networks.	Chapter 1 and Chapter 2 of "Neural Networks and Deep Learning
Lecture 7, Bayes Networks.	Murphy (UBC) notes, first few pages are most relevant
Lecture 8, Hidden Markov Models.	Stanford notes
Lecture X, Convolutional Neural Networks.
Lecture 9, Learning Theory.