About Me

• Machine Intelligence / Startups / Finance
• • Moved from NYC to Singapore in Sep-2013
• 2014 = 'fun' :
• • Machine Learning, Deep Learning, NLP
  • Robots, drones
• Since 2015 = 'serious' :: NLP + deep learning
• • GDE ML; TF&DL co-organiser
  • Red Dragon AI...

About Red Dragon AI

• Google Partner : Deep Learning Consulting & Prototyping
• SGInnovate/Govt : Education / Training
• Research : NeurIPS / EMNLP
• Products :
• • Conversational Computing
  • Natural Voice Generation - multiple languages
  • Knowledgebase interaction & reasoning

Outline

• whoami = DONE
• Stuff about TensorFlow.js
• Stuff about Faces & Deep Learning
• Demo (everyone!)
• Wrap-up

TensorFlow.js

• Foundations
• Low-level JS
• Importing from Keras
• High-level JS

TF.js foundations

• deeplearning.js → Open Source on GitHub
• Includes GPU processing!
• • ... (mis-)using WebGL
• See it up close

TF.js direct

• import * as tf from '@tensorflow/tfjs-core';
• Low level primatives ++
• See it up close

Use existing models

Import from Keras !

# Python
import tensorflowjs as tfjs
def train(...):
    model = keras.models.Sequential()   # for example
    #...
    model.fit(...)
    tfjs.converters.save_keras_model(model, tfjs_target_dir)  
    
# JS
import * as tf from '@tensorflow/tfjs';
const model = await tf.loadModel('https://foo.bar/tfjs_artifacts/model.json');
const example = tf.fromPixels(webcamElement);  // for example
const prediction = model.predict(example);

Even Higher Level

<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>
  
// Step 1: Create an image classifier with MobileNet
const classifier = ml5.imageClassifier('MobileNet', onModelReady);

// Step 2: select an image
const img = document.querySelector("#myImage")

// Step 3: Make a prediction
let prediction = classifier.predict(img, gotResults);

// Step 4: Do something with the results!
function gotResults(err, results) {
  console.log(results);
  // all the amazing things you'll add
}

Why Faces?

• Looking for something visual...
• Everyone has a face
• Everyone has a browser :
• • Demo should work on laptop Browser
  • And slowly on Mobile
• Hands-on with controversial AI...

Face tasks

• Find the face in the image
• Work out orientation of the face
• Do stuff with 'registered' image
• For example :
• • Emotion Detection
  • Face Recognition

Training emotions

• Training set
• • Face as pixels
  • Emotion 'category' as output
• Train from scratch ...
• • ... or use Transfer Learning

Training positioning

• Training set
• • Face as pixels
  • Relative positions of 'points of interest'
• Segment image into patches :
• • Each patch returns guess of
  • where the 'corners' are relative to it
  • + some kind of voting scheme

Training positioning

• As a picture (sourced from these slides)

Training recognition

• People look different from each other
• Want to represent them as vectors :
• • Easily detect similarity
  • ... by using 'dot-product'
• We're doing Self-Supervised learning...
• More like word2vec embeddings

Training recognition

• As a picture (credit to Raúl Gómez)

About the Demo

• Based on face-api.js:
• • Code on GitHub
  • And blog post(s)
  • ... be careful of API creep

The Demo

• https:// reddragon.ai / demo / faces
• Please follow along :
• • Open up Laptops!
  • Check on your Phone...
  • • 'Open as website'
    • 'Allow camera'

Demo

Bounding Box(es)

Demo

Capture Embedding

Demo

My Embeddings

Demo

T-SNE

Deep Learning
MeetUp Group

• MeetUp.com / TensorFlow-and-Deep-Learning-Singapore
• Next Meeting:
• • Date : 19-December, hosted at Google
• Typical Contents :
• • Talk for people starting out
  • Something from the bleeding-edge
  • Lightning Talks

Deep Learning : Jump-Start Workshop

• First part of Full Developer Course
• Dates : January 15, 16 + online
• • 2 week-days + online content
  • Play with real models & Pick-a-Project
  • Certificates, etc
• Cost is heavily subsidised for SC/PR!
• SGInnovate - Talent - Talent Development -
  Deep Learning Developer Series

Deep Learning
Developer Course

• Module #1 : JumpStart (see previous slide)
• Each 'module' includes :
• • Instruction
  • Individual Projects
  • 70%-100% funding via IMDA for SG/PR
• Module #2 : (est) Feb 2020 : Advanced Computer Vision
• Module #3 : (est) March 2020 : Advanced NLP
• Location : SGInnovate/BASH