TensorFlow
from Source

TensorFlow & Deep Learning SG

Martin Andrews @ redcatlabs.com
Martin Andrews @ reddragon.ai

21 September 2017

About Me

  • Machine Intelligence / Startups / Finance
    • Moved from NYC to Singapore in Sep-2013
  • 2014 = 'fun' :
    • Machine Learning, Deep Learning, NLP
    • Robots, drones
  • Since 2015 = 'serious' :: NLP + deep learning
    • & Papers...
    • & Dev Course...

TensorFlow Install


pip install tensorflow
#  or
pip install tensorflow-gpu
   

TensorFlow Install (better)


virtualenv -p python3 --system-site-packages ./env3-tensorflow
. env3-tensorflow/bin/activate
# CLI-prompt changes
pip install --upgrade pip

pip install tensorflow
#  or
pip install tensorflow-gpu
   

TensorFlow.new

  • To get all the 'cool stuff' :
    • Need to install from source
    • Be prepared for headaches

TensorFlow Staging

  • Code goes through a staging process:
    • (ideas / code from everywhere ... CLA)
    • Google Devs
    • GitHub master (/branch)
    • Release Candidates
    • pip release

Are you sure?

  • Be prepared to :
    • Install a new build tool
    • Big download(s)
    • Long rebuilds
    • (GPU) Fight both Nvidia and Google
    • Play with cool stuff
  • If contributing : Sign CLA

TF steps

  • Install bazel
  • Create/use blank Python virtualenv
  • Download tensorflow from GitHub
  • ./configure build options
  • bazel build the code
  • Create the .whl file
  • Install using pip

Why bazel?

  • Poor-man's blaze (Google's build environment)
  • Can support huge projects
  • Aggressive caching of pre-built components
  • Configuration via 'simple' BUILD files

Install bazel


wget https://github.com/bazelbuild/bazel/releases/  \
      download/0.5.4/bazel-0.5.4-installer-linux-x86_64.sh
# This downloads 185Mb ...

chmod +x bazel-0.5.4-installer-linux-x86_64.sh 

# This will install it to ~/bin/ which is Ok, since 
# ```which missing-binary``` shows is in that path
./bazel-0.5.4-installer-linux-x86_64.sh --user 

# Seems to unpack stuff..

bazel version
#Build label: 0.5.4
   

Create virtualenv


virtualenv-3.6 --system-site-packages env3-tensorflow
. ./env3-tensorflow/bin/activate
   

Download tensorflow


git clone https://github.com/tensorflow/tensorflow   # Downloads ~120Mb
# This downloads 120Mb ...

cd tensorflow
   

Run ./configure


./configure

You have bazel 0.5.4 installed.
Please specify the location of python. 
[Default is / ... /env3-tensorflow/bin/python]: 
...
Do you wish to build TensorFlow with Hadoop File System support? [y/N]: 
No Hadoop File System support will be enabled for TensorFlow.
...
Do you wish to build TensorFlow with XLA JIT support? [y/N]: Y
XLA JIT support will be enabled for TensorFlow.
...
   

Build using bazel


bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

# Lots of downloads...
# including protobuf, llvm, ...

# INFO: Elapsed time: 5218.471s, Critical Path: 86.01s
   

  • Add --config=cuda for GPU code (cross-fingers)
  • Add --copt=-O for compilation of boringssl (?!?)
  • Build broken after 2 out of 5 git pulls in last 2 weeks

Build/install python .whl


bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

pip install /tmp/tensorflow_pkg/tensorflow-*.whl
   

Test the install


cd SOMEWHERE-ELSE
python

import tensorflow as tf

a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

print(sess.run(c))
   

Hidden stuff?


# in tensorflow.python.eager.context (at end) :

# TODO(apassos): This should not be a part of the public API.

from tensorflow.python.util.all_util import remove_undocumented
remove_undocumented(__name__)
   

Hidden stuff exposed

Adding to tensorflow/python/util/all_util.py lets us see :


    #print("Deleting : ", extra_symbol, " from ", fully_qualified_name)
    print("Deleting : ", fully_qualified_name)
    
Deleting :  tensorflow.contrib.stateless.remove_undocumented
Deleting :  tensorflow.contrib.stat_summarizer.remove_undocumented
Deleting :  tensorflow.contrib.decision_trees.remove_undocumented
Deleting :  tensorflow.contrib.timeseries.remove_undocumented
Deleting :  tensorflow.contrib.seq2seq.remove_undocumented
Deleting :  tensorflow.contrib.sparsemax.remove_undocumented
Deleting :  tensorflow.contrib.tpu.remove_undocumented
Deleting :  tensorflow.contrib.tfprof.remove_undocumented
   

Wrap-up

  • TensorFlow install from source can be painful
  • But if you need the latest stuff...
GitHub - mdda

* Please add a star... *

Deep Learning
MeetUp Group

8-week Deep Learning
Developer Course

  • Actual : Start 25 September
  • Twice-Weekly 3-hour sessions will include :
    • Instruction
    • Individual Projects
  • Location : SGInnovate
  • Status : FULL

3 Lightning Talks

  • Here are a few things that popped up recently :
    • Salesforce's CoVe
    • 3d Face reconstruction
    • GoogleBrain Reddit AMA

CoVe

  • Improve word embeddings by learning to translate
    • eg: GloVe enhanced by disambiguating layer
  • Parallel language text 'cheap' compare to annotated text
  • Makes language lessons valuable again


https://einstein.ai/research/
learned-in-translation-contextualized-word-vectors/
= blog, paper, code & models

3d Face Reconstruction

  • CNN to learn volumetric 'depth'
  • Easy to think of cool applications

3d Volumetric reconstruction
http://www.cs.nott.ac.uk/~psxasj/3dme/
= demo, video, paper & code

Google Brain Reddit AMA

  • Google Brain opens up to Reddit
  • More 'crunchy' than last year

GoogleBrain reddit AMA
https://www.reddit.com/r/MachineLearning/comments/
6z51xb/we_are_the_google_brain_team_wed_love_to_answer/

3 Lightning Talks

  • That wasn't so hard, was it?
  • Simply share some enthusiasm! Audience = ♥
  • Just 'suggest' using MeetUp suggestion link

- QUESTIONS -


Martin.Andrews @
RedCatLabs.com

Martin.Andrews @
RedDragon.AI


My blog : http://blog.mdda.net/

GitHub : mdda