Bahman Moraffah - Books

Probability, Statistics, and Random Processes with Applications in Learning Theory

Bahman Moraffah, 2021
Probability theory provides a principled, practical, mathematical approach for learning theory. This book provides the foundamental background for doing research in machine learning, engineering, and statistics. It provides a unified treatment of theratical and practical aspenct of probability theory. The treatment is comprehensive and self-contained with many applications, targeted at researchers and students in machine learning, engineering, and applied statistics.

Contents:
Part I: Probability Theory

• Probability Space

• Continuous and Discrete Random Variables

• Multiple Random Variables

• Parametric Point Estimation

• Probability Theory: Applications

• Convergence and Asymptotic Behavior
  

Part II: Random Processes

• Introduction to Random Processes

• Markov Processes

• Poisson Processes
  

Part III: Applications in Learning Theory

• Probability and its Application in Machine Learning

• Bootstrap and Monte Carlo Resampling Methods

• Information Geometry and its Applications in Machine Learning

Bayesian Modeling and Inference: A Bayesian Approach to Machine Learning

Bahman Moraffah, 2022
It provides a comprehensive treatment of advanced learning thoery. It spans a wide range of statistical tools from frequntists to Bayesian. This book extensively emphesizes on the aadvances in machine learning learning theory from Bayesian perspective. This book can be used for an advanced graduate course in Bayesian statistics.
Contents:

• Preliminary

  • Measure theoretic definition

  • Stochastic convergences

  • Information theory

  • Group theory

  • Conceptual things

  • Exponential family

  • Gaussian

• Theory of Point Estimation

  • Likelihood and first order methods such as MLE

  • Delta method

  • Asymptotic statistics

  • M and Z estimators

  • U statistics

  • Empirical processes

  • L-Stattiitcs

  • Parametric minimax theory

  • Expectation-minimization algorithm to compute

• Introduction to nonparametric Statistics

  • Concentration of measure

  • Density estimation

  • Minimax theory

• Statistical Decision Theory: A frequentist Approach

• Bootstrap

• Information Geometry

• Regression

  • Linear regression

• Introduction to Bayesian Statistics

  • Bayesian statistics: introduction and example — Bayes rule, Exchangeability, posterior, inference, marginalization

  • Singel parameter estimation

  • Multi- parameter estimation

  • Choice of prior

  • Frequentist property of Bayesian modeling

• Advanced Posterior Computation

  • Markov chain Monte Carlo method

  • Gibbs sampling — efficient Gibbs sampling

  • Metropolis-Hastings algorithm

  • Building MC algorithm

  • Variational Bayes

• Hierarchical Modeling

• Mixture Model

• Robust inference

• Bayesian Regression

  • Linear Models

  • Generalized linear model

  • Linear regression — Bayesian linear model, Model selection — Bayesian comparison, Gibbs sampling and model averaging

• Bayesian Decision theory

• Poisson Processes

  • Poisson distribution and relationship to multinomial and binomial distribution

  • Definition of Poisson process

  • Campbell’s theorem

• Bayesian nonparametrics I

  • Dirichlet process – gamma process

  • Hierarchical Dirichlet process

  • Dependent Dirichlet process

  • Two-parameter Poisson-Dirichlet process

• Bayesian nonparametrics II

  • Beta processes, stick-breaking, and power law

  • Indian Buffett process

  • Hierarchical Indian buffet proceses

• Bayesian nonparametric III

  • Completely random measure

  • Normalized random measure

  • Kingman paintbox

  • Feature allocation and paintbox

  • Truncated random measure

• Bayesian Nonparameteric IV

  • Gaussian Processes

  • Learning Functions and Gaussian Processes

  • Bayesian Optimization

• Bayesian Network and Causal Inference

  • Directed Graphical model and Inference

  • Causal inference

• Undirected graphical model

  • Definitions

  • Inference on Undirected graphs