HOUSE_OVERSIGHT_026397.jpg

networks

A machine learning program that can learn how to play an Atari game without any human supervision or hand-crafted engineering (the feat that gave DeepMind 500M from Google) now just takes about 130 lines of Python code.

These models do not have interesting motivational systems, and a relatively simple architecture. They currently seem to mimic some of the stuff that goes on in the first few layers of the cortex. They learn object features, visual styles, lighting and rotation in 3d, and simple action policies. Almost everything else is missing. But there is a lot of enthusiasm that the field might be on the right track, and that we can learn motor simulations and intuitive physics soon. (The majority of the people in AI do not work on this, however. They try to improve the performance for the current benchmarks.)

Noam's criticism of machine translation mostly applies to the Latent Semantic Analysis models that Google and others have been using for many years. These models map linguistic symbols to concepts, and relate concepts to each other, but they do not relate the concepts to "proper" mental representations of what objects and processes look like and how they interact. Concepts are probably one of the top layers of the learning hierarchy, i.e. they are acquired *after* we learn to simulate a mental world, not before. Classical linguists ignored the simulation of a mental world entirely.
It seems miraculous that purely conceptual machine translation works at all, but that is because concepts are shared between speakers, so the structure of the conceptual space can be inferred from the statistics of language use. But the statistics of language use have too little information to infer what objects look like and how they interact.

My own original ideas concern a few parts of the emerging understanding of what the brain does. The "request-confirmation networks" that I have introduced at a NIPS workshop in last the December are an attempt at modeling how the higher layers might self-organize into cognitive programs.

Cheers!

Joscha

--
please note
The information contained in this communication is
confidential, may be attorney-client privileged, may
constitute inside information, and is intended only for
the use of the addressee. It is the property of
JEE
Unauthorized use, disclosure or copying of this
communication or any part thereof is strictly prohibited
and may be unlawful. If you have received this
communication in error, please notify us immediately by
return e-mail or by e-mail to jeevacation@gmail.com, and
destroy this communication and all copies thereof,
including all attachments. copyright -all rights reserved

HOUSE_OVERSIGHT_026397