This seminar course will expand on the concepts introduced in CSC 535. The primary aim of this course is to provide students an introduction to advanced techniques in probabilistic graphical models (PGMs) and statistical machine learning (ML) and the ability to apply those techniques to their own research. In particular, students will learn to perform statistical inference and reasoning in statistical models where standard techniques are computationally prohibitive. The course will survey state-of-the-art ML research including: exponential families, variational inference, advanced Markov chain Monte Carlo sampling, Bayesian nonparametrics, Bayesian optimization, and Bayesian Deep Learning. Upon conclusion of this course students will be capable of developing new methods and advancing the state-of-the-art in ML and PGM research.
To successfully complete this course, students should have the following skills:
Instructor: Jason Pacheco, GS 724, Email: pachecoj@cs.arizona.edu Office Hours: Tuesday, 10:00-11:30am Course Homepage: http://www.pachecoj.com/courses/csc665-1 D2L: https://d2l.arizona.edu/d2l/home/814136 Instructor Homepage: http://www.pachecoj.com
Each class will combine instructor lectures with student presentations of assigned readings. In addition each student must present the results of a term project of their choosing during the final period.
The primary objective of this course is to provide students a deeper understanding and familiarity with advanced topics in PGMs and statistical ML, as well as recent research in these areas. Our primary focus will be algorithms for Bayesian posterior inference in hierarchical statistical models. We will cover, both, sample-based MCMC approaches as well as approximate methods based on variational inference.
Planned topics:
Upon conclusion of this course, students will have learned to read, understand, and critique machine learning research articles. Students will demonstrate their understanding of articles through class presentations and critical summaries. Through assigned readings, students will be familiar with state-of-the-art research in key areas of statistical machine learning and PGMs. Students will apply these techniques to their own research, which will be assessed through term projects.
The UA’s policy concerning Class Attendance, Participation, and Administrative Drops is available at http://catalog.arizona.edu/policy/class-attendance-participation-and-administrative-drop
The UA policy regarding absences for any sincerely held religious belief, observance or practice will be accommodated where reasonable: http://policy.arizona.edu/human-resources/religious-accommodation-policy.
Absences preapproved by the UA Dean of Students (or dean’s designee) will be honored. See https://deanofstudents.arizona.edu/absences.
Attendance and participation in class discussion is required at all lectures. As CSC 655-1 is a seminar course, all of the material benefit occurs in-class. Failure to attend lectures will affect the student’s final course grade. If you anticipate being absent or are unexpectedly absent, please contact the instructor as soon as possible. To request a disability-related accommodation to this attendence policy, please contact the Disability Resource Center at (520) 621-3268 or drc-info@email.arizona.edu. If you are experiencing unexpected barriers to your success in courses, the Dean of Students Office is a central support resource for all students and may be helpful. The Dean of Students Office is located in the Robert L. Nugent Building, room 100, or call (520) 621-7057.
Online communication will be conducted using D2L and official UA email addresses.
Required readings will be made available electronically, and on a weekly basis, as the course progresses.
Access to computing equipment with Matlab, Python, or other similar programming environment for completion of term project.
Each student will select from one assigned reading to present in a 60-minute format to the class. Each student will complete a term project of their choosing, and subject to instructor approval.
Students will present their term projects during the final exam period.
For the U. Arizona final exam regulations see: https://www.registrar.arizona.edu/courses/final-examination-regulations-and-information
For the U. Arizona final exam schedule see: http://www.registrar.arizona.edu/schedules/finals.htm
Students shall submit a critical summary for each assigned reading. To receive full credit, summaries must demonstrate that the student has adequately read and critiqued the material. Each student will additionally select among assigned papers and prepare a 1hr presentation to the class in which they explain key technical details of the reading. To receive full credit for class participation, students must attend and participate in the discussion of all classes. Students should contact the instructor regarding absences for make-up. Finally, term project grading will be assessed based on how well the idea is conceived, planned, executed, and presented.
Grading Breakdown:
| Attendance / participation: | 10% |
| Paper presentation: | 20% |
| Critical reading summaries: | 20% |
| Term project proposal: | 10% |
| Term project (presentation and writeup): | 40% |
Requests for incomplete (I) or withdrawal (W) must be made in accordance with University policies, which are available at http://catalog.arizona.edu/policy/grades-and-grading-system#incomplete and http://catalog.arizona.edu/policy/grades-and-grading-system#Withdrawal, respectively.
Week 1: Course mechanics / Introduction to Bayesian inference + PGM primer
Week 2: Hierarchical models
Week 3: Exponential families
Week 4: Variational inference (Mean field variational Bayes)
Week 5: Variational inference (Loopy belief propagation + Expectation propagation)
Week 6: Advanced MCMC (Hamiltonian MCMC) / Term project proposals due
Week 7: Advanced MCMC (Parallel tempering)
Week 8: Bayesian nonparametrics (Gaussian processes)
Week 9: Bayesian nonparametrics (Dirichlet process)
Week 10: Bayesian nonparametrics (Hierarchical Dirichlet process)
Week 11: Bayesian optimization
Week 12: Bayesian deep learning
Week 13: Term project presentations
The Department of Computer Science is committed to providing and maintaining a supportive educational environment for all. We strive to be welcoming and inclusive, respect privacy and confidentiality, behave respectfully and courteously, and practice intellectual honesty. Disruptive behaviors (such as physical or emotional harassment, dismissive attitudes, and abuse of department resources) will not be tolerated. The complete Code of Conduct is available on our department web site. We expect that you will adhere to this code, as well as the UA Student Code of Conduct, while you are a member of this class.
To foster a positive learning environment, students and instructors have a shared responsibility. We want a safe, welcoming, and inclusive environment where all of us feel comfortable with each other and where we can challenge ourselves to succeed. To that end, our focus is on the tasks at hand and not on extraneous activities (e.g., texting, chatting, reading a newspaper, making phone calls, web surfing, etc.).
Students are asked to refrain from disruptive conversations with people sitting around them during lecture. Students observed engaging in disruptive activity will be asked to cease this behavior. Those who continue to disrupt the class will be asked to leave lecture or discussion and may be reported to the Dean of Students.
The UA Threatening Behavior by Students Policy prohibits threats of physical harm to any member of the University community, including to oneself. See http://policy.arizona.edu/education-and-student-affairs/threatening-behavior-students.
The Disability Resources Offices provides guidelines regarding accessibility and accommodations: http://drc.arizona.edu/instructors/syllabus-statement.
Students are encouraged to share intellectual views and discuss freely the principles and applications of course materials. However, graded work/exercises must be the product of independent effort unless otherwise instructed. Students are expected to adhere to the UA Code of Academic Integrity as described in the UA General Catalog. See http://deanofstudents.arizona.edu/academic-integrity/students/academic-integrity.
The University Libraries have some excellent tips for avoiding plagiarism, available at http://new.library.arizona.edu/research/citing/plagiarism.
The University is committed to creating and maintaining an environment free of discrimination; see http://policy.arizona.edu/human-resources/nondiscrimination-and-anti-harassment-policy
UA Academic policies and procedures are available at http://catalog.arizona.edu/policies
Student Assistance and Advocacy information is available at http://deanofstudents.arizona.edu/student-assistance/students/student-assistance
Information contained in the course syllabus, other than the grade and absence policy, may be subject to change with advance notice, as deemed appropriate by the instructor.
The University of Arizona sits on the original homelands of indigenous peoples who have stewarded this land since time immemorial. Aligning with the university’s core value of a diverse and inclusive community, it is an institutional responsibility to recognize and acknowledge the people, culture, and history that make up the Wildcat community. At the institutional level, it is important to be proactive in broadening awareness throughout campus to ensure our students feel represented and valued.