Abstract of FKI-222-97

Document-Name:  pub/fki/fki-222-97.ps.gz
Authors:	Sepp Hochreiter 
		Juergen Schmidhuber 
Revision-Date:	1997/06/18 
Category:	Technical Report (Forschungsberichte Künstliche Intelligenz)
Abstract:	Traditional approaches to sensory coding and unsupervised 
		learning ignore the information-theoretic complexity of the 
		code-generating mapping. Ours does not.  It is called 
		``low-complexity coding and decoding'' (Lococode). Lococodes
		generated by Lococode (1) convey information about the input 
		data, (2) can be computed from the data by a low-complexity 
		mapping (LCM), (3) can be decoded by a LCM.  We implement 
		Lococode by training autoassociators with Flat Minimum Search,
		a recent, general method for finding low-complexity neural 
		nets. Experiments show: lococodes are based on feature 
		detectors (FDs), never unstructured, usually sparse, sometimes
		factorial or local (depending on the data). Although its
		objective function does not contain an explicit term enforcing
		sparse or factorial codes, Lococode extracts optimal codes for
		difficult versions of the ``bars'' benchmark problem. Unlike, 
		e.g., ICA, it does not need to know the number of independent 
		data sources.  It produces familiar, biologically plausible 
		FDs when applied to real world images. As a preprocessor for 
		a vowel recognition benchmark problem it sets the stage for 
		excellent classification performance.
Keywords:	coding, complexity, lococode, FMS, sparse codes, ICA, MDL,
		feature detectors, preprocessing, sensory coding, 
		unsupervised learning
Size:		19 pages
Language:	English
ISSN:		0941-6358
Copyright:	The ``Forschungsberichte Künstliche Intelligenz''
		series includes primarily preliminary publications,
		specialized partial results, and supplementary
		material. In the interest of a subsequent final
		publication these reports should not be copied. All
		rights and the responsability for the contents of the
		report are with the authors, who would appreciate
		critical comments.

Gerhard Weiss