Description of Course

This seminar course will expand on the concepts introduced in CSC 535. The primary aim of this course is to provide students an introduction to advanced techniques in probabilistic graphical models (PGMs) and statistical machine learning (ML) and the ability to apply those techniques to their own research. In particular, students will learn to perform statistical inference and reasoning in statistical models where standard techniques are computationally prohibitive. The course will survey state-of-the-art ML research including: exponential families, variational inference, advanced Markov chain Monte Carlo sampling, Bayesian nonparametrics, Bayesian optimization, and Bayesian Deep Learning. Upon conclusion of this course students will be capable of developing new methods and advancing the state-of-the-art in ML and PGM research.

Course Prerequisites or Co-requisites

To successfully complete this course, students should have the following skills:

  • Understanding of probability: distributions, marginalization, Bayes’ rule
  • Familiarity with probabilistic graphical models: Markov random fields, factor graphs, Bayes nets
  • At least an undergraduate level understanding of linear algebra, calculus, and some familiarity with concepts in nonlinear and discrete optimization
  • Advanced programming (to complete term project)

Instructor and Contact Information:

Instructor: Jason Pacheco, GS 724, Email:
Office Hours: Tuesday, 10:00-11:30am
Course Homepage:
Instructor Homepage:

Course Format and Teaching Methods

Each class will combine instructor lectures with student presentations of assigned readings. In addition each student must present the results of a term project of their choosing during the final period.

Course Objectives

The primary objective of this course is to provide students a deeper understanding and familiarity with advanced topics in PGMs and statistical ML, as well as recent research in these areas. Our primary focus will be algorithms for Bayesian posterior inference in hierarchical statistical models. We will cover, both, sample-based MCMC approaches as well as approximate methods based on variational inference.

Planned topics:

  • Introduction to Bayesian inference + PGM primer
  • Hierarchical modeling with examples (e.g. latent Dirichlet allocation, state-space models, etc.)
  • Variational Inference:
    • Exponential families
    • Mean field variational Bayes
    • (Loopy) Belief Propagation
    • Expectation Propagation
  • Advanced Markov chain Monte Carlo
  • Bayesian Nonparametrics:
    • Gaussian Processes
    • Dirichlet Process / Dirichlet Mixture Model
    • Hierarchical Dirichlet Process
  • Bayesian Optimization
  • Bayesian Deep Learning

Expected Learning Outcomes

Upon conclusion of this course, students will have learned to read, understand, and critique machine learning research articles. Students will demonstrate their understanding of articles through class presentations and critical summaries. Through assigned readings, students will be familiar with state-of-the-art research in key areas of statistical machine learning and PGMs. Students will apply these techniques to their own research, which will be assessed through term projects.

Absence and Class Participation Policy

The UA’s policy concerning Class Attendance, Participation, and Administrative Drops is available at

The UA policy regarding absences for any sincerely held religious belief, observance or practice will be accommodated where reasonable:

Absences preapproved by the UA Dean of Students (or dean’s designee) will be honored. See

Attendance and participation in class discussion is required at all lectures. As CSC 655-1 is a seminar course, all of the material benefit occurs in-class. Failure to attend lectures will affect the student’s final course grade. If you anticipate being absent or are unexpectedly absent, please contact the instructor as soon as possible. To request a disability-related accommodation to this attendence policy, please contact the Disability Resource Center at (520) 621-3268 or If you are experiencing unexpected barriers to your success in courses, the Dean of Students Office is a central support resource for all students and may be helpful. The Dean of Students Office is located in the Robert L. Nugent Building, room 100, or call (520) 621-7057.

Course Communications

Online communication will be conducted using D2L and official UA email addresses.

Required Texts or Readings

Required readings will be made available electronically, and on a weekly basis, as the course progresses.

Required or Special Materials (if any)

Access to computing equipment with Matlab, Python, or other similar programming environment for completion of term project.

Assignments and Examinations: Schedule/Due Dates

Each student will select from one assigned reading to present in a 60-minute format to the class. Each student will complete a term project of their choosing, and subject to instructor approval.

Final Examination or Project

Students will present their term projects during the final exam period.

For the U. Arizona final exam regulations see:

For the U. Arizona final exam schedule see:

Students shall submit a critical summary for each assigned reading. To receive full credit, summaries must demonstrate that the student has adequately read and critiqued the material. Each student will additionally select among assigned papers and prepare a 1hr presentation to the class in which they explain key technical details of the reading. To receive full credit for class participation, students must attend and participate in the discussion of all classes. Students should contact the instructor regarding absences for make-up. Finally, term project grading will be assessed based on how well the idea is conceived, planned, executed, and presented.

Grading Breakdown:

Attendance / participation: 10%
Paper presentation: 20%
Critical reading summaries: 20%
Term project proposal: 10%
Term project (presentation and writeup): 40%

Requests for incomplete (I) or withdrawal (W) must be made in accordance with University policies, which are available at and, respectively.

Scheduled Topics/Activities

Week 1: Course mechanics / Introduction to Bayesian inference + PGM primer
Week 2: Hierarchical models
Week 3: Exponential families
Week 4: Variational inference (Mean field variational Bayes)
Week 5: Variational inference (Loopy belief propagation + Expectation propagation)
Week 6: Advanced MCMC (Hamiltonian MCMC) / Term project proposals due
Week 7: Advanced MCMC (Parallel tempering)
Week 8: Bayesian nonparametrics (Gaussian processes)
Week 9: Bayesian nonparametrics (Dirichlet process)
Week 10: Bayesian nonparametrics (Hierarchical Dirichlet process)
Week 11: Bayesian optimization
Week 12: Bayesian deep learning
Week 13: Term project presentations

Department of Computer Science Code of Conduct

The Department of Computer Science is committed to providing and maintaining a supportive educational environment for all. We strive to be welcoming and inclusive, respect privacy and confidentiality, behave respectfully and courteously, and practice intellectual honesty. Disruptive behaviors (such as physical or emotional harassment, dismissive attitudes, and abuse of department resources) will not be tolerated. The complete Code of Conduct is available on our department web site. We expect that you will adhere to this code, as well as the UA Student Code of Conduct, while you are a member of this class.

Classroom Behavior Policy

To foster a positive learning environment, students and instructors have a shared responsibility. We want a safe, welcoming, and inclusive environment where all of us feel comfortable with each other and where we can challenge ourselves to succeed. To that end, our focus is on the tasks at hand and not on extraneous activities (e.g., texting, chatting, reading a newspaper, making phone calls, web surfing, etc.).

Students are asked to refrain from disruptive conversations with people sitting around them during lecture. Students observed engaging in disruptive activity will be asked to cease this behavior. Those who continue to disrupt the class will be asked to leave lecture or discussion and may be reported to the Dean of Students.

Threatening Behavior Policy

The UA Threatening Behavior by Students Policy prohibits threats of physical harm to any member of the University community, including to oneself. See

Accessibility and Accommodations

The Disability Resources Offices provides guidelines regarding accessibility and accommodations:

Code of Academic Integrity

Students are encouraged to share intellectual views and discuss freely the principles and applications of course materials. However, graded work/exercises must be the product of independent effort unless otherwise instructed. Students are expected to adhere to the UA Code of Academic Integrity as described in the UA General Catalog. See

The University Libraries have some excellent tips for avoiding plagiarism, available at

UA Nondiscrimination and Anti-harassment Policy

The University is committed to creating and maintaining an environment free of discrimination; see

Additional Resources for Students

UA Academic policies and procedures are available at

Student Assistance and Advocacy information is available at

Confidentiality of Student Records

Subject to Change Statement

Information contained in the course syllabus, other than the grade and absence policy, may be subject to change with advance notice, as deemed appropriate by the instructor.

The University of Arizona sits on the original homelands of indigenous peoples who have stewarded this land since time immemorial. Aligning with the university’s core value of a diverse and inclusive community, it is an institutional responsibility to recognize and acknowledge the people, culture, and history that make up the Wildcat community. At the institutional level, it is important to be proactive in broadening awareness throughout campus to ensure our students feel represented and valued.

© Jason Pacheco, 2019