Future of Mind

SG Futurists

Martin Andrews @ redcatlabs.com

16 August 2015

About Me

  • Machine Intelligence / Startups / Finance
    • Moved to Singapore in Sep-2013
  • Last year (2014) = 'fun' :
    • Machine Learning, Deep Learning, NLP
    • Robots, drones
    • "MeetUp Pro"
  • This year (2015) = 'serious' NLP

The past

  • Chess (1950s - 1997)
  • Jeopardy (2004 - 2011)
  • Driverless cars (1980s - present)

Chess

  • 1957 : Computer to #1 'within 10 years'
  • 1978 : First game win against master
  • 1997 : First match win against #1 (Kasparov)
    • DeepBlue : 200MM moves per second
  • 2009 : Grandmaster level on your phone
    • 'Pocket Fritz' : 20k moves/sec

Jeopardy

  • 2004 : New problem identified by IBM
  • 2008 : IBM approached show hosts
  • 2011 : IBM Watson wins
Jeopardy

"4Tb of disk storage"

Driverless cars

  • 1980s : Early work (CMU and BMW/Munich)
  • 2004 : DARPA GC1 - 12km furthest
  • 2005 : DARPA GC2 - 90% of contestents > 12km
  • 2009 : Google Car driving autonomously around CA
  • 2015 : Google Car :
    • Driven 2 million miles
    • 14 minor accidents (caused by humans)

Current Hype ::
Deep Learning

  • Neural Networks with many layers
  • Same as original Neural Networks in 1980s/1990s
  • Deliberate branding by leading lights in mid 2000s
    • Hinton, Bengio, Le Cun, Andrew Ng

Single "Neuron"

Different weights compute different functions

One Neuron

"Neural Networks"

Layers of neurons combine and
can form more complex functions

Multi-Layer

"Deep Neural Networks"

Google ImageNet

GoogLeNet (2014)

"It's Different this Time"

  • Data : More effective than expected
  • GPUs : Training networks since 2012
  • New Techniques (a.k.a. Tricks)

Data : More effective than expected

  • >1Bn Photos per day
  • Wikipedia (~6Gb) standard training unit
Photo Uploads

Internet growth beating Moore's Law

GPUs : Huge speed-ups

  • 1000s of cores, >1 GFLOP per dollar
  • NVidia GPU vs CPU

New Techniques

  • Sigmoids → ReLUs
  • Dropout (worse is better)
  • Batch normalization
  • Networks for the whole stack
  • Benchmark Competitions

What can be done now

  • Speech recognition
  • Language translation
  • Vision :
    • Object recognition
    • Automatic captioning
  • Reinforcement Learning

Speech Recognition

Android feature since Jellybean (v4.3, 2012) using Cloud

Trained in ~5 days on 800 machine cluster

Speech Recognition

Embedded in phone since Android Lollipop (v5.0, 2014)

Translation

Google's Deep Models are on the phone

Google Translate

"Use your camera to translate text instantly in 26 languages"

Translations for typed text in 90 languages

House Numbers

Google Street-View (and ReCaptchas)

House Numbers

Better than human

ImageNet Results

ImageNet Results

(now human competitive on ImageNet)

Captioning Images

Labelling Results

Some good, some not-so-good

Reinforcement Learning

Google's DeepMind purchase

Learn to play games from the pixels alone

DeepMind Atari

Better than humans 2 hours after switching on

"A.I. Effect"

Arrival of Smart machines

  • Can roughly calibrate 'smarts' :
    • spider, mouse, dog
    • Homer Simpson, Bart Simpson, Lisa Simpson
    • Einstein, etc
  • Not clear that twice the capacity ⇒ twice the smarts

How far can Deep Learning take us?

  • Lots of fruit now found to be 'low hanging'
  • But it feels like building a better ladder ...
    • ... when the goal is to reach the Moon

Main Interest : Strong A.I.

  • Question :
    • What is Strong A.I. ?
  • Approximate Answer :
    • Difference between Lisa and Bart
  • Problem :
    • Where to start...

Scientific Method

  • Compare : Science of ...
    • ... Gravity
    • ... Love
    • ... Consciousness

Philosophical Questions

  • Not going too deep...
  • Is "Strong A.I." actually possible ?
  • How do we measure progress / success ?

Magic and the Brain

  • Standard objection : Brains are special
    • Homunculus : The real 'you'
  • Electrification : One neuron at a time

Turing Test

  • Test for humaness through interaction
  • Chatbots are getting better...
    • Cheating is now an issue
  • Looking for Lisa Simpson qualities (not Homer)

Searle's Chinese Room

  • Complex process might appear intelligent
    • Simulation of understanding vs real understanding
    • Relates to Consciousness more than Intelligence
  • Decent Responses :
    • Question is just "Does a submarine swim?"
    • System is what counts, not components

Problems with studying Consciousness

  • Difficult to dig into the actual processes
  • Conscious thought is like the surface of water ...
    • ... whereas the mind is the ocean itself

'Thought process' of A.I.

  • ... likely to be very different from humans
  • Simple vs Hard problems
    • People whose names have 'M' as first letter
    • People whose names have 'R' as third letter
  • Incompatible hardware
    • Uploading doesn't make sense to me

Higher principles of A.I.

  • Illustrative example :
    • Aeroplanes don't flap their wings
    • What is the analogue of aerodynamics for intelligence?

Searching for Higher Principles

  • Using Biological neurons:
    • we know that brain solves problems well
    • but that doesn't mean it is the best approach
  • What are the take-aways from current successes?
  • What actual problems can be solved more abstractly?

Stuff that works better-than-expected

  • The Unreasonable Effectiveness of Data
  • Convolutional networks for images
    • ... and back-track to actual brains
  • Word Embedding
    • ... language models from text alone
  • Transfer learning
    • Learning one task can 'cross-fertilize'

Stuff that can be abstracted

  • Hebbian Learning
    • "Cells that fire together wire together"
  • ~ Optimisation
    • Gradient descent
    • Genetic Algorithms
  • ~ Principal Component Analysis
    • Matrix Methods

Other Candidates for Higher Principles

  • Sparse representations
  • Learning as compression :
    • Novelty / Surprise detection
    • Minimal Description Length / Occam's razor
  • Bayesian statistics
  • Language itself is revealing
    • Highly evolved protocol

Futurism

  • Graph of silicon vs brain compute power (omitted)
  • Singularity enthusiast / skeptic :
    • Different 'grades' of machine are different problems
    • Smart Police for Smart Criminals
    • Perhaps A.I. workers will arrive in time to defuse the demographic time-bomb

Which Companies are involved?

  • 'Company X' wants A.I. to :
    • IBM : Sell to Enterprise
    • Apple : be Magical
    • Baidu : be Functional
    • Facebook : be your ~ Friend
    • Google : Happen

Conclusions

  • Great strides being made
    • But A.I. Effect is irksome
  • Deep Learning frontier is being pushed by competitions
    • Need for a common 'Strong A.I.' competition
    • ... but don't know the right questions to ask

- QUESTIONS -


Martin.Andrews @
RedCatLabs.com


My blog : http://mdda.net/

GitHub : mdda