About Me

• Machine Intelligence / Startups / Finance
• • Moved from NYC to Singapore in Sep-2013
• 2014 = 'fun' :
• • Machine Learning, Deep Learning, NLP
  • Robots, drones
• Since 2015 = 'serious' :: NLP + deep learning
• • & GDE ML; TF&DL co-organiser
  • & Papers...
  • & Dev Course...

About Red Dragon AI

• Google Partner : Deep Learning Consulting & Prototyping
• SGInnovate/Govt : Education / Training
• Products :
• • Conversational Computing
  • Natural Voice Generation - multiple languages
  • Knowledgebase interaction & reasoning

Outline

• whoami = DONE
• Neural Structured Learning
• • Outline
  • Quick walkthrough (concrete graph)
  • Synthetic graphs
• Notebook : Adversarial robustness
• Wrap-up

Neural Structured Learning

Outline

• Have a regular classification task
• Examples are also related :
• • .. to each other (labelled)
  • .. to unlabelled examples

Relations via Graph

• Enhance internal representation with graph hints

Losses via Graph

• Enhance internal representation with mathematics

Benefits of NSL

• Reduce labelled data required
• Make use of loose relationships
• Clusters aid interpretability

NSL Building

• Standard classification model
• Encode graph relationships
• • Seems to need TFRecords
• Wrap original model ⇒ Graph-aware
• Train Graph-aware version
• Use model (no performance penalty)

Neural Structured Learning

'Straight' Application

• Known graph (given explicitly):
• • CORA paper classification (with citations)
• Dataset of 2708 scientific publications :
• • Classified into seven classes
  • 1433 unique words to learn over
  • Citation network = 5429 links

Neural Structured Learning

'Synthetic' Application

• Create graph on the fly:
• • IMDB reviews (with 'neighbours')
• Augment the IMDB dataset :
• • 25,000 reviews for training
  • 25,000 reviews for testing
  • Create graph of nearest neighbours
  • ... defined by document embedding

Adversarial Images

• Generate 'related' images
• ... designed to be mis-classified
• Use NSL to provide fix-up
• Improved version of Google's example

Deep Learning
MeetUp Group

• MeetUp.com / TensorFlow-and-Deep-Learning-Singapore
• Next Meeting:
• • Date : Early December, hosted at Google
• Typical Contents :
• • Talk for people starting out
  • Something from the bleeding-edge
  • Lightning Talks

Deep Learning : Jump-Start Workshop

• First part of Full Developer Course
• Dates : November 13, 14 + online
• • 2 week-days + online content
  • Play with real models & Pick-a-Project
  • Certificates, etc
• Cost is heavily subsidised for SC/PR!
• SGInnovate - Talent - Talent Development -
  Deep Learning Developer Series

Deep Learning
Developer Course

• Module #1 : JumpStart (see previous slide)
• Each 'module' includes :
• • Instruction
  • Individual Projects
  • 70%-100% funding via IMDA for SG/PR
• Module #2 : (est) Feb 2020 : Advanced Computer Vision
• Module #3 : (est) March 2020 : Advanced NLP
• Location : SGInnovate/BASH