Abstract of FKI-198-94
Title: On Learning How to Learn Learning Strategies
Authors: Juergen Schmidhuber
Revision-Dates: 1994/11/24, 1995/01/31
Category: Technical Report (Forschungsberichte Künstliche Intelligenz)
Abstract: This paper introduces the ``incremental self-improvement paradigm''.
Unlike previous methods, incremental self-improvement encourages a
reinforcement learning system to improve the way it learns, and to
improve the way it improves the way it learns, without significant
theoretical limitations -- the system is able to ``shift its induc-
tive bias'' in a universal way. Its major features are: (1) There
is no explicit difference between ``learning'', ``meta-learning'',
and other kinds of information processing. Using a Turing machine
equivalent programming language, the system itself occasionally
executes self-delimiting, initially highly random ``self-modifi-
cation programs'' which modify the context-dependent probabilities
of future programs (including future self-modification programs).
(2) The system keeps only those probability modifications computed
by ``useful'' self-modification programs: those which bring about
more payoff per time than all previous self-modification programs.
(3) The computation of payoff per time takes into account all the
computation time required for learning -- the entire system life is
considered: boundaries between learning trials are ignored (if there
are any). A particular implementation based on the novel paradigm is
presented. It is designed to exploit what conventional digital
machines are good at: fast storage addressing, arithmetic operations
etc. Experiments illustrate the system's mode of operation.
Keywords: self-improvement, self-reference, machine learning,
reinforcement learning, introspection
Size: 20 pages
Copyright: The ``Forschungsberichte Künstliche Intelligenz''
series includes primarily preliminary publications,
specialized partial results, and supplementary
material. In the interest of a subsequent final
publication these reports should not be copied. All
rights and the responsability for the contents of the
report are with the authors, who would appreciate