Probabilistic Modeling and Statistical Inference

Abstraction: Bayesian and frequentist foundations for principled statistical inference

Key points:

• Decision making formalizes as selecting the action maximizing utility under uncertainty; inference quantifies which model configurations are consistent with observations
• The true data generating process is unknown; inference operates within a "model configuration space" (small world subset of all distributions), never the full space
• Observational variation arises from both aleatoric (ontological) and epistemic sources; both are modeled identically as probability distributions over an observation space
• Frequentist inference uses binary consistency assignments (estimators) calibrated by worst-case expected utility across all model configurations, avoiding probability over parameters
• Bayesian inference encodes domain expertise as a prior distribution over the model configuration space and updates it with observed data via likelihood
• George Box: "all models are wrong but some are useful" — careful generative model design (phenomenon / environment / probe layers) is essential

Connections: Michael Betancourt · Bayesian Inference · Statistical Modeling · Probabilistic Modeling · Frequentist Inference

Source: https://betanalpha.github.io/assets/case_studies/modeling_and_inference.html