CISC 5800: Machine Learning

CISC 5800: Machine Learning

Class times: Monday, 5:30 – 7:45pm, LL 914
Instructor: Prof. Daniel D. Leeds (my homepage)
Office: LL 819D for Office Hours; normally JMH 332 (Rose Hill)
E-mail:
Office hours: Monday 4-5pm

Full syllabus is here.

Course text: No book will be required, but the following will be useful as references.

"Pattern Recognition and Machine Learning" by Bishop
"Machine Learning: A Probabilistic Approach" by Murphy

Programming: We will have programming assignments throughout the semester. I will require you complete your programming assignments in Matlab. There are several ways to use Matlab/Matlab-equivalent-software:

You can run Matlab in the computer lab or (without graphics) by remote login to our department machines.
You can download your own student license for $50-$100.
You can download and use Octave , which is a free version of Matlab that is somewhat less powerful and can be tricky to install, depending on your operating system.

Sections below:

Resources
Announcements
Slides
Assignments

Resources:
Computing guides
Linux Commands - important Linux commands for working on erdos
vi Commands - important commands for the vi text editor; you are welcome to use emacs instead of vi
A Guide to Putty - Information for Windows users on accessing erdos
Note, to access erdos on Mac: (1) Open terminal, (2) type "ssh username@erdos.dsm.fordham.edu" , inserting your username before the @
Matlab
Extra background on Matlab
Matlab programming practice — download the accompanying data file sampleData.mat and function file newFunc.m
Calculus
Calculus practice - some optional problems with derivatives
Midterm practice
Practice questions
Practice questions answers

Midterm Answers

76-90 A range

64-76 B range

52-64 C range

Final practice
Practice questions -- additional questions two lines below
Practice answers
Additional practice, coded in colors red, green, blue
Additional practice answers BLUE
Additional practice answers GREEN
Additional practice answers RED

Announcements:
Nov 21, 1:40am: The final will be held December 18.
October 11, 5:00am I am offline 5pm Oct 11 to around 9pm Oct 14. I will reply Saturday night to e-mails sent during my offline time.
Oct 4, 2:30am: The midterm will be held October 30.
Sept 1, 12:15pm: Our first class meeting will take place Wednesday September 7 (Monday schedule). I will not be able to make it to class, but Lisha Zhou will be there to present our first week of material. I also will be missing my official ``office hours'' before class that day Sept 7.

Slides: Coming soon... Check slides after lectures as well for updates made during class!
I have not found any one textbook or online resource to be an optimal match for the material we cover at the mathematical level we cover it. However, Andrew Ng's lecture notes, available on Stanford's Machine Learning course web site, often can be a helpful online read. I recommend these notes as well as looking through one of our course textbooks.

Supplementary reading

Lecture 1, Course logistics, background math, intro to classifiers.
Spoken version: Part 1 Part 2 Part 3 Part 4

Lecture 1.5, Matlab intro.
spoken version

Lecture 2, Bayes classifier. Ng notes 2 particularly pages 8-11

Lecture 3, Logistic classifier. Ng Notes 1 Elements of Part II (starting on page 16)

Lecture 4, Support vector machines. Ng notes 3 parts of pages 1-20

Lecture 5, Dimensionality reduction. Ng notes 10 on PCA and Ng notes 11 on ICA

Lecture 6, Neural networks. Chapter 1 and Chapter 2 of "Neural Networks and Deep Learning

Lecture 7, Bayes Networks. Murphy (UBC) notes, first few pages are most relevant

Lecture 8, Hidden Markov Models Stanford notes

Lecture Extra, Convolutional Neural Nets There may be an extra credit question on this.

Assignments:
Homework 0 due Sept 18, corrected Sept 11 evening

Homework 0 answers

82-91 "A range"

73-82 "B range"

Homework 1 due Sept 25 (written) and Sept 29 (code); hw1data.mat

Homework 1 answers now available
Homework 2 - due October 16 (Parts A and B), and October 20 (Part C); data available as hw2data.mat

Homework 2 answers now available for parts A and B

POINTS FOR TOTAL HW2 SCORES

70-80 "A range"

60-70 "B range"

50-60 "C range"

Final project - due December 13 --- extension from December 8

Homework 3 - due November 27 (Parts A and B), and December 1 (Part C); data available as mnist_train.csv; supplemental code available as commonDirection.m (Matlab) or commonDirection.py (Python)

Homework 3 answers now available for parts A and B

60.5 - 71 A range

53 - 60.5 B range

46 - 53 C range

Homework 4 - OPTIONAL, due December 11

Homework 4 answers

91 - 106 A range

79 - 91 B range

67 - 79 C range

55 - 67 D range

	Supplementary reading
Lecture 1, Course logistics, background math, intro to classifiers. Spoken version: Part 1 Part 2 Part 3 Part 4
Lecture 1.5, Matlab intro. spoken version
Lecture 2, Bayes classifier.	Ng notes 2 particularly pages 8-11
Lecture 3, Logistic classifier.	Ng Notes 1 Elements of Part II (starting on page 16)
Lecture 4, Support vector machines.	Ng notes 3 parts of pages 1-20
Lecture 5, Dimensionality reduction.	Ng notes 10 on PCA and Ng notes 11 on ICA
Lecture 6, Neural networks.	Chapter 1 and Chapter 2 of "Neural Networks and Deep Learning
Lecture 7, Bayes Networks.	Murphy (UBC) notes, first few pages are most relevant
Lecture 8, Hidden Markov Models	Stanford notes
Lecture Extra, Convolutional Neural Nets	There may be an extra credit question on this.