Abstract of FKI-198-94

Document-Name:  fki-200-94.ps.gz
Title:		On Learning How to Learn Learning Strategies
Authors:	Juergen Schmidhuber 
Revision-Dates:	1994/11/24, 1995/01/31 
Category:	Technical Report (Forschungsberichte Künstliche Intelligenz)
Abstract:  This paper introduces the ``incremental self-improvement paradigm''.
           Unlike previous methods,  incremental self-improvement  encourages a
           reinforcement learning  system to improve  the way it learns, and to
           improve the way  it improves the way it learns,  without significant
           theoretical limitations -- the system  is able to ``shift its induc-
           tive bias'' in  a universal way.  Its major features are:  (1) There
           is no  explicit difference  between ``learning'', ``meta-learning'',
           and other kinds of  information  processing.  Using a Turing machine
           equivalent  programming  language,  the system  itself  occasionally
           executes  self-delimiting,  initially  highly  random ``self-modifi-
           cation programs'' which modify  the context-dependent  probabilities
           of  future  programs (including future  self-modification programs).
           (2) The system keeps  only those probability  modifications computed
           by ``useful'' self-modification  programs:  those which  bring about
           more payoff per time  than all previous  self-modification programs.
           (3) The computation  of payoff per time  takes into account  all the
           computation time required for learning -- the entire system  life is
           considered: boundaries between learning trials are ignored (if there
           are any). A particular implementation based on the novel paradigm is
           presented.  It is  designed to  exploit  what  conventional  digital
           machines are good at: fast storage addressing, arithmetic operations
           etc. Experiments illustrate the system's mode of operation.
Keywords:	self-improvement, self-reference, machine learning,
                reinforcement learning, introspection
Size:		20 pages
Language:	English
ISSN:		0941-6358
Copyright:	The ``Forschungsberichte Künstliche Intelligenz''
		series includes primarily preliminary publications,
		specialized partial results, and supplementary
		material. In the interest of a subsequent final
		publication these reports should not be copied. All
		rights and the responsability for the contents of the
		report are with the authors, who would appreciate
		critical comments.


Gerhard Weiss