Description
- Abstract:
- Many nonparametric Bayesian models can be viewed as an infinite-dimensional limit of a family of finite-dimensional models. However, another way to construct a flexible Bayesian model is to take the same family and put a prior on the dimension --- that is, to use a variable-dimension model --- for example, putting a prior on the number of components in a finite mixture. Using theory and experiments, this thesis analyzes some of the differences and similarities between the nonparametric and variable-dimension approaches, develops new inference algorithms for these models, and explores new variable-dimension models. Primarily, we focus on the Dirichlet process mixture (DPM) and a variable-dimension alternative that we refer to as the mixture of finite mixtures (MFM) model. One of the main differences between DPMs and MFMs is the behavior of the posterior on the number of clusters. We show that for a large class of nonparametric mixtures, including DPMs and Pitman--Yor process mixtures over a wide range of families of component distributions, the posterior on the number of clusters does not concentrate at the true number of components when the data comes from a finite mixture. Meanwhile, it is known that the MFM posterior on the number of components concentrates at the true number, assuming the model is correctly specified. We explore the properties of the MFM, finding that it has many of the same attractive features as the DPM: a simple partition distribution, exchangeability properties, restaurant process, random discrete measure representation, and in certain special cases, a simple stick-breaking representation. As a result, many of the same approximate inference algorithms used for nonparametric mixtures can be easily adapted to the MFM. We also propose two new variable-dimension models: the hierarchical mixture of finite mixtures (HMFM) as an alternative to the hierarchical Dirichlet process (HDP), and the mixture of finite feature models (MFFM) as an alternative to the Indian buffet process (IBP). As with the MFM, these variable-dimension models exhibit some of the same appealing characteristics as their nonparametric counterparts.
- Notes:
- Thesis (Ph.D. -- Brown University (2014)
Access Conditions
- Rights
- In Copyright
- Restrictions on Use
- Collection is open for research.
Citation
Miller, Jeffrey W.,
"Nonparametric and Variable-Dimension Bayesian Mixture Models: Analysis, Comparison, and New Methods"
(2014).
Applied Mathematics Theses and Dissertations.
Brown Digital Repository. Brown University Library.
https://doi.org/10.7301/Z0T1520K
Relations
Collection:
-
Applied Mathematics Theses and Dissertations
Theses and Dissertations for the Applied Mathematics department....