Going Deeper :
Transfer Learning

TensorFlow & Deep Learning SG

martin @ reddragon.ai


21 November 2017


About Me

  • Machine Intelligence / Startups / Finance
    • After PhD, went into Finance as a Quant
    • Moved from NYC to Singapore in Sep-2013
  • 2014 = 'fun' :
    • Machine Learning, Deep Learning, NLP
  • Since 2015 = 'serious' :: NLP + deep learning
    • SG funded project : SiTF AI Gold
    • Google Developer Expert
    • Deep Learning Dev Course
    • & Papers...

Overview

"Something more challenging"

  • Use a state-of-the-art TensorFlow model
  • Solve a problem it wasn't trained on
  • ... Deep Learning as a component

Goals

  • Distinguish between Classic and Modern sports cars
    • Could be other types of images
    • Could be many different classes
  • Very small training time
  • Very small number of training examples
  • Able to be put into production

Image Classification

  • ImageNet Competition
  • over 15 million labeled high-resolution images...
  • ... in over 22,000 categories

ImageNet Karpathy

AlexNet

AlexNet Representations

ImageNet Classification with Deep Convolutional Neural Networks (2012) (Alex Krizhevsky)

CNN Representations

  • Each successive layer ...
    • ... seems to learn 'higher level' representations
  • All created by BackProp (no hand-crafted features)
  • Surprisingly like the actual brain

Early Levels

Layers 1 and 2

Visualizing and Understanding Convolutional Networks (2013) (large download)

Middle Level

Layer 3

Later Levels

Layers 4 and 5

2014 : State-of-the-Art

Google Inception-v1

GoogLeNet - aka Inception v1

Approach

  • Take an existing model trained for ImageNet
  • Instead of softmax layer :
    • use the logits layer as features to train an SVM
  • Support Vector Machines are 'well understood' tools

Network Picture

Repurposed Network

The Code

Conclusions

  • Really works
  • Didn't have to train a deep neural network
  • Could plug this TF model into existing pipeline
GitHub - mdda

* Please add a star... *

- QUESTIONS -


Martin.Andrews @
RedDragon.ai


My blog : http://mdda.net/

GitHub : mdda

Our Next MeetUp

  • "TensorFlow and Deep Learning Singapore"
    • Hosted by Google
  • 5-December-2017 : ~ Strata Conf
  • Different from usual :
    • Googlers from Mountain View
    • TensorFlow News and Teasers

Regular MeetUps

  • "TensorFlow and Deep Learning Singapore"
    • Hosted by Google
    • And by SGInnovate
  • Recommence in Jan-2018
    • Something for beginners
    • Something more challenging
    • Lightning talks

8-week Deep Learning
Developer Course

  • Actual : Mid-Sept to Mid-Nov
  • Twice-Weekly 3-hour sessions included :
    • Instruction
    • 3 structured projects
    • 2 self-directed projects
  • Cost: S$3000 - supported by WSG
  • Everybody worked hard...

Deep Learning : Beginner Course

  • Dates + Cost : TBA ::
    • Full day (week-end)
    • Play with real models
    • Get inspired!
    • Pick-a-Project to do at home
    • 1-on-1 support online
    • Regroup on a week-night
  • http://bit.ly/2zVXtRm