ECE and CS 8725 Supervised Learning - University of Missourivigir.ee.missouri.edu/~gdesouza/ece8725/ECE-CS8725-course-info.pdf · The University of Missouri - Columbia Electrical

The University of Missouri - Columbia Electrical & Computer Engineering Department ECE8725 Supervised Learning

F20 ECE & CS 8725 Syllabus Dr. DeSouza Page 1

ECE and CS 8725 Supervised Learning

Prerequisite: CS/ECE 4720/7720 or instructor consent

G. DeSouza

August 2020

Vision-Guided and Intelligent Robotics Lab Electrical & Computer Engineering Department

University of Missouri - Columbia Columbia, MO 65211



Description: This course introduces the theories and applications of advanced supervised machine learning methods. It covers hidden Markov model and expectation maximization (EM) algorithms, probabilistic graphical models, non-linear support vector machine and kernel methods. The course emphasizes both the theoretical underpinnings of the advanced supervised learning methods and their applications in the real world. Instructor: Prof. Guilherme N. DeSouza TA: N/A G. DeSouza’s Office Hours: By appointment or drop-ins - NAKA 325 E-Mail: [email protected] Course Texts:

C.M. Bishop. Pattern Recognition and Machine Learning, Springer, 2006. ISBN: 978-0-387-31073-2

References:

Papers from PAMI, AI, MLJ, JMLR, PR and various others will be handed out over the semester.

Course Format: Lectures will occur in-person on Mondays, Wednesdays, and Fridays as advertise on MyZou. Technology permitting, Zoom streaming will be available to those who contract COVID-19, but no recording – i.e. ‘attendance’ at the time of the lecture is still required. There will be one midterm exam, plus one final project. Homework assignments (small projects) and paper reviews using a mock review site will be posted on the web and collected approximately every other week. CANVAS is NOT the only form of communication/announcement/storage of information for this course. Lecture Notes and Assignments: Students should NOT count on availability of lecture notes. That is, students should take their own notes. However, class notes are USUALLY made available also on the web. Homework assignments will be made available in advance on the web at http://web.missouri.edu/~desouzag/ (follow the link for ECE&CS8725). Course Topics: (not necessarily in chronological order) 1. Hidden Markov models and EM algorithm

a. Markov models b. Hidden Markov models c. Three main problems of HMM d. Maximum likelihood for HMM e. Forward-backward algorithm f. Sum-product algorithm for the HMM g. Viterbi algorithm h. Baum-Welch algorithm i. EM algorithms for HMM j. Scaling factors



2. Graphical Models

a. Bayesian networks b. Conditional independence c. D-separation d. Markov random fields e. Factorization properties f. Inference on graphical models g. Learning the graph structure

3. Support vector machines (focused on non-linear SVM) and kernel methods

a. VC dimension and structural risk minimization b. Large margin classifier c. Dual representation and Lagrange optimization d. Soft-margin classifier e. Kernel trick for non-linear SVM f. Multi-class SVM g. SVM for regression h. Relevance vector machines i. Constructing kernels j. Gaussian processes

4. Semi-Supervised Learning

a. Generative models b. Low-density separation c. Graph-based methods d. Heuristic approaches

5. Optimization

6. Deep Learning

a. Convolutional Neural Networks b. Recurrent NN c. Applications: Vision, NLP, etc…



Homework/Small Projects: Homework will be in the form of small projects with two distinct parts: practical and theoretical questions. It will be assigned approximately once every two weeks. Course Policies: Cheating is strictly prohibited. Cheating violates any concept of honesty, integrity, and engineering ethics and it shall not be tolerated. Any evidence of copying or plagiarism, partial or in full, is considered cheating. All parts caught cheating shall: 1) receive a score of 0; 2) be turned over to the Department Chairman and the Academic Provost; and 3) face the appropriate penalties established by the University, including the possibility of being expelled. Grade Construction

GRADE COMPONENT POINTS Exam 100 (midterm) Student Paper Presentation 100 Homework/Paper Review 100 Final Project 100 (Project and Presentation) Total Points 400

The midterm exam may be cancelled and extra projects may be assigned instead in the case of campus shutdown.



COURSE TIME SCHEDULE:

Fall 2020 HW1 HW2 HW3 HW4 HW5 Final Project

Week 1 8/24

Week 2 8/31 Assigned

Week 3 9/7

Week 4 9/14 Due Assigned

Week 5 9/21


Week 7 10/5

Mid-Term – Oct 7th


Week 9 10/19

Week10 10/26 Due Assigned

Week11 11/2

Week12 11/9 Due

Week13 11/16

Thanksgiving Break (Nov. 23 - Nov. 27)

Week15 11/30 Student Project Presentations

Week16 12/7 Reports Due

FINAL EXAM: TBA



Statement on Academic Honesty: Academic integrity is fundamental to the activities and principles of a university. All members of the academic community must be confident that each person's work has been responsibly and honorably acquired, developed, and presented. Any effort to gain an advantage not given to all students is dishonest whether or not the effort is successful. The academic community regards breaches of the academic integrity rules as extremely serious matters. Sanctions for such a breach may include academic sanctions from the instructor, including failing the course for any violation, to disciplinary sanctions ranging from probation to expulsion. When in doubt about plagiarism, paraphrasing, quoting, collaboration, or any other form of cheating, consult the course instructor. According to University policy, instructors are required to inform students of specific guidelines regarding cheating in their courses. Instructors are required by University policy to report incidents of cheating to the Office of the Provost. In compliance with this rule, all incidents of cheating by students in this course will be reported to the Office of the Provost for determination of possible disciplinary action. Any student found to have cheated during an exam or lab will be given an “F” grade for the class and the evidence will be sent to the Provost's Office. Students submitting the same or similar solutions to a programming homework will be given a 0 for the assignment and the evidence will be sent to the Provost's Office for determination of possible disciplinary action. Second occurrences of cheating in a homework will lead to an “F” grade for the class. Unless an assignment is specifically structured as a group project, duplicate homework written in collaboration with others is not acceptable. Although it is permissible to discuss the homework with others, these discussions should be of a general nature. All work at a detailed level must be done on your own. Students submitting the same or similar solutions to the homework will be considered as having cheated. No statements or actions made by anyone can alter this policy. Statement on ADA: Students with Disabilities: If you anticipate barriers related to the format or requirements of this course, if you have emergency medical information to share with me, or if you need to make arrangements in case the building must be evacuated, please let me know as soon as possible. If disability related accommodations are necessary (for example, a note taker, extended time on exams, captioning), please establish an accommodation plan with the Disability Center (disabilitycenter.missouri.edu, S5 Memorial Union, 573- 882-4696), and then notify me of your eligibility for reasonable accommodations. For other MU resources for persons with disabilities, visit ada.missouri.edu.



COVID-19 Related Information:

Syllabus statement regarding “Decreasing the Risk of COVID-19 in Classrooms and Labs” (https://provost.missouri.edu/syllabus-information/)

See above regarding in-person class meetings

How students must contact me, both regularly and in the case of an emergency

Social distancing and face-covering protocols are required for all F2F components of this course. Students should come to class with their own face covering, but that they will be available for students who don’t have one. Students should also know ahead of time that instructors will contact the Dean of Students if students are not wearing one (and do not have an accommodation for not wearing one).

See above regarding course attendance policy that addresses students who are unable to come to class for medically confirmed infection by COVID. In essence, due to the nature of the course, all students must participate in class discussions.

Accommodation Plan Please note that instructors will not be contacted with specific student health information, but public health officials and contact tracers will be aware of positive cases and will be working with affected individuals. (If an instructor or student has been exposed to COVID-19, they will be contacted by a contact tracer.) In other words, for this course, students must disclose why they need to work remotely. Pivot Plan If it becomes necessary to move the course fully online, course will remain the same, with the exceptions that: meetings will be live, via Zoom; and the mid-term exam will be canceled and replaced by projects, as mentioned above.

Documents

ECE and CS 8725 Supervised Learning - University of Missourivigir.ee.missouri.edu/~gdesouza/ece8725/ECE-CS8725-course-info.pdf · The University of Missouri - Columbia Electrical