Probabilistic Machine Learning: Advanced Topics

Key links

• Short table of contents
• Long table of contents
• Preface
• Draft pdf of the main book, 2024-06-26. CC-BY-NC-ND license. (Please cite the official reference below.)
• Supplementary material
• Issue tracker.
• Code to reproduce most of the figures
• Acknowledgements
• Endorsements

 @book{pml2Book,
 author = "Kevin P. Murphy",
 title = "Probabilistic Machine Learning: Advanced Topics",
 publisher = "MIT Press",
 year = 2023,
 url = "http://probml.github.io/book2"
}

Downloads since 2022-02-28.

Table of contents

Endorsements

• "Kevin Murphy has distilled the vast and confusing literature on machine learning and neural networks into a beautifully written and extremely clear textbook that will be a wonderful resource for both new students and seasoned researchers who are trying to keep up with this fast moving field. The chapter on generative models is a masterpiece." -- Geoff Hinton, U. Toronto/ Google.

• "Kevin Murphy had already impressed and greatly benefited the machine learning community with his introductory book on probabilistic ML and I am delighted to see the depth and breadth of material in his new sequel on advanced probabilistic ML. The book covers topics which I believe are at the heart of past and upcoming advances in our field, while often lacking in the training of graduate students in computer science, and I therefore recommend it highly to all of them." -- Yoshua Bengio, U. Montreal

• "This book is an amazing tour de force: Murphy and his co-authors have described and systematized virtually all of the important advances in machine learning over the past 30 years. Pick any topic, and they provide a crisp description of the state-of-the-art methods in a common, well-chosen notation and using a set of core concepts in modeling, statistics, and optimization. This book will be a valuable starting point for students entering the field and a wonderful reference for seasoned researchers. It is hard to imagine a better antidote to the vast, confusing, and voluminous literature in machine learning. This is such an amazing book! I learned things even in the sections where I’m fairly up to date with the literature. " --- Tom Dietterich, Oregon State University

• "The prior version of Dr. Murphy's book was amongst the 3-4 books I recommended to students and machine learning colleagues at all levels as being essential to own, and, perhaps more importantly, to always have readily at hand. As machine learning has matured and evolved, no other comprehensive resource of this nature has even remotely kept pace with modern methodological developments. This new version is a blessing for the machine learning community and frankly, at this moment in time, is the only truly necessary machine learning book to own. Very few people in the world can do what Dr. Murphy has done here and the world owes him its thanks." -- Frank Wood, UBC.

• "Whether teaching machine learning to undergrads, master students, or PhD students, I found myself time and time again choosing the 2012 "Machine Learning: A Probabilistic Perspective" as the primary textbook. When I heard about the new "Probabilistic Machine Learning" series, I was thrilled to see the expanded and modernized set of topics; this will be the go-to book for my ML courses at Stanford. Kevin Murphy has a phenomenal ability to go deep while making topics digestible to a broad audience. His writing is clear and concise with great visuals throughout. I highly recommend this as "the book" for anyone wanting to become a well-versed ML expert." -- Emily Fox, Stanford.

• "Murphy’s book is certainly the most comprehensive resource on machine learning available today. With the growing body of research in the field, it is a daunting challenge to provide an organized perspective of the current state of knowledge. This book achieves this feat by integrating classic material, like MCMC inference, with very recent developments like denoising diffusion models. The material is organized and presented in a very accessible and intuitive manner, making the book an asset for any researcher or practitioner in the field." -- Amir Globerson, Tel Aviv University.

• "This new Advanced Topics volume will provide a crucial path from the foundations of machine learning to state-of-the-art probabilistic models, many of which have only been described in the research literature. I particularly like the way it draws parallels between variational and Monte Carlo inference, and between graphical models and (Bayesian) deep learning, so one can see the deep links between methods that are sometimes cast as competitors. I look forward to the next generations of probabilistic machine learning that this volume inspires." --- Erik Sudderth, UC Irvine.

• "This book provides an outstanding and deep tour of the most foundational ideas in probabilistic machine learning. It explains the essential mathematical and computational tools that a student needs to move beyond the basics. With this book, Murphy provides a comprehensive resource that will be great not just to learn from, but also as a reference to return to again and again." -- Ryan Adams, Princeton.

• "Kevin Murphy's book is a landmark achievement in machine learning. It provides an in-depth coverage of a wide range of topics in probabilistic machine learning, from inference methods to generative models and decision making. It gives a modern perspective on these topics, bringing them up to date with recent advances in deep learning and representation learning. The insights in this book are essential for a solid understanding of the field, and I highly recommend it to students and experts alike." -- Zoubin Ghahramani, Cambridge/Google.

• "As a researcher trained by Murphy's 2012 book, I was excited to read its sequel "Probabilistic Machine Learning: Advanced Topics." This new book brings us to the forefront of cutting-edge probabilistic ML, distilling the recent advances into a systemic exposition. It communicates the core ideas of these recent advances in an impressively clear and intuitive way, while managing to convey the depth of these ideas by situating them in a broad context. It will become a major reference that I constantly return to." -- Yixin Wang, U. Michigan.

Acknowledgements

• People who helped to write various sections and chapters: Alex Alemi (Google), Jeff Bilmes (U. Washington), Peter Chang, Marco Cuturi (Apple, work done at Google), Alexander D'Amour (Google), Finale Doshi-Velez (Harvard), Roy Frostig (Google), Justin Gilmer (Google), Giles Harper-Donnelly, Been Kim (Google), Durk Kingma (Google), Simon Kornblith (Google), Balaji Lakshminarayanan (Google), Lihong Li (Amazon, work done at Google), Xinglong Li (UBC), Shakir Mohamed (Deepmind), George Papamakarios (Deepmind), Zeel Patel (IIT Gandhinagar), Ben Poole (Google), Mihaela Rosca (Deepmind / UCL), Vinayak Rao (Purdue), Yang Song (Stanford), Victor Veitch (Google / U. Chicago), Andrew Wilson (NYU).
• Many people who helped make or improve the figures, including: Vishal Ghoniya, Ankita Kumari Jain, Aleyna Kara, Zeel Patel, Karm Patel, Dhruv Patel, Nitish Sharma, Mahmoud Soliman.
• Participants in the Google Summer of Code (GSOC) for 2021, including Ming Liang Ang, Aleyna Kara, Gerardo Duran-Martin, Srikar Reddy Jilugu, Drishti Patel, and co-mentor Mahmoud Soliman.
• Participants in the Google Summer of Code (GSOC) for 2022, including Peter Chang, Giles Harper-Donnelly, Xinglong Li, Zeel Patel, Karm Patel, Qingyao Sun, and co-mentors Nipun Batra and Scott Linderman.
• Many other people who contributed code (see auto-generated list here).
• Many people who helped with proof reading (see book preface for list).