#135 Bayesian Calibration and Model Checking, with Teemu Säilynoja

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

• Intro to Bayes Course (first 2 lessons free)
• Advanced Regression Course (first 2 lessons free)

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!

Visit our Patreon page to unlock exclusive Bayesian swag ;)

Takeaways:

• Teemu focuses on calibration assessments and predictive checking in Bayesian workflows.
• Simulation-based calibration (SBC) checks model implementation
• SBC involves drawing realizations from prior and generating prior predictive data.
• Visual predictive checking is crucial for assessing model predictions.
• Prior predictive checks should be done before looking at data.
• Posterior SBC focuses on the area of parameter space most relevant to the data.
• Challenges in SBC include inference time.
• Visualizations complement numerical metrics in Bayesian modeling.
• Amortized Bayesian inference benefits from SBC for quick posterior checks. The calibration of Bayesian models is more intuitive than Frequentist models.
• Choosing the right visualization depends on data characteristics.
• Using multiple visualization methods can reveal different insights.
• Visualizations should be viewed as models of the data.
• Goodness of fit tests can enhance visualization accuracy.
• Uncertainty visualization is crucial but often overlooked.

Chapters:

09:53 Understanding Simulation-Based Calibration (SBC)

15:03 Practical Applications of SBC in Bayesian Modeling

22:19 Challenges in Developing Posterior SBC

29:41 The Role of SBC in Amortized Bayesian Inference

33:47 The Importance of Visual Predictive Checking

36:50 Predictive Checking and Model Fitting

38:08 The Importance of Visual Checks

40:54 Choosing Visualization Types

49:06 Visualizations as Models

55:02 Uncertainty Visualization in Bayesian Modeling

01:00:05 Future Trends in Probabilistic Modeling

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden