Changes between Version 3 and Version 4 of ML/Principles


Ignore:
Timestamp:
Feb 5, 2018, 12:21:23 PM (2 years ago)
Author:
schiptsov
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ML/Principles

    v3 v4  
    11= First principles for Machine Learning =
    22
    3 Learning, in principle, is an ability to develop due to repeated exposure to various aspects of reality (through sensory input) an inner representation which reflects various properties of ''what is'' and then use (pattern match against) this inner representation to achieve better performance in the same shared environment.
     3Learning, in principle, is an ability to develop due to repeated exposure to various aspects of reality (through sensory input generalized as experience) an inner representation which reflects various properties of ''what is'' and to use pattern matching against this inner representation to achieve better performance in the same (or similar) shared environment.
    44
    55''A computer program is said to learn from experience {{{E}}} with respect to some task {{{T}}} and performance measure {{{P}}}, if its performance at task {{{T}}}, as measured by {{{P}}}, improves with experience {{{E}}}''.
    66
    7 There are three major areas in machine learning.
     7The inner representation must necessarily have a ''structure'' which reflects or represents various constraints or properties of the environment. The better reflection (less "distance" from what is) the better representation. The ''structure'' could be physical (made out of neurons, axons and dendrites) or virtual, made out of ones and zeroes in a computer memory. The principle is the same - ''an inner structure that matches outer structure as best as it could''.
     8
     9This does not mean, of course, that the inner structure must match the outer structure of the universe up to last atom. To the contrary, it should, at least in theory, use as economical (smallest in terms of matter and energy consumption, but good-enough) representation as possible. The most important criteria is that representation must be free from capturing of what is not there (like to ignore things like, say, shadows of a tree, which are very complex but useless phenomena). The inner representation must match reality only. Any highest animal has a surprisingly adequate, good-enough representation of its environment (habitat).
     10
     11In other words, it is a building and using an adequate ''map'' of what is, given that ''a map is NOT a territory''. It must be accurate in the first place, and detailed-enough. No bullshit (or explicitly marked as bullshit). This, by the way, is what a good education supposed to accomplish.
    812
    913= Backpropagation =