Abstract of FKI-205-95

Document-Name:  fki-205-95.ps.gz
Title:          Improved Gaussian Mixture Density Estimates Using Bayesian
                Penalty Terms and Network Averaging
Authors:        Dirk Ormoneit and Volker Tresp
Revision-Date:  7/1/95
Category:	Technical Report (Forschungsberichte Künstliche Intelligenz)
Abstract:       We compare two regularization methods which can be used to 
                improve the generalization capabilities of Gaussian mixture 
                density estimates. The first method consists of defining a 
                Bayesian prior distribution on the parameter space. We derive 
                EM (Expectation Maximization) update rules  which  maximize 
                the a posterior parameter probability in contrast to the usual
                EM rules for Gaussian mixtures which  maximize the likelihood 
                function. In the second approach we apply ensemble averaging 
                to density estimation. This includes Breiman's "bagging", which
                has recently been found to produce impressive results for
                classification networks. To our knowledge this is the first 
                time that ensemble averaging is applied to improve density 
                estimation.
Keywords:       Gaussian Mixtures, EM algorithm, Bagging, Averaging
Size:		12 pages
Language:	English
ISSN:		0941-6358
Copyright:	The ``Forschungsberichte Künstliche Intelligenz''
		series includes primarily preliminary publications,
		specialized partial results, and supplementary
		material. In the interest of a subsequent final
		publication these reports should not be copied. All
		rights and the responsability for the contents of the
		report are with the authors, who would appreciate
		critical comments.

Gerhard Weiss