Piano Transcription


TensorFlow & Deep Learning SG


Martin Andrews @ reddragon.ai

27 September 2018

About Me

  • Machine Intelligence / Startups / Finance
    • Moved from NYC to Singapore in Sep-2013
  • 2014 = 'fun' :
    • Machine Learning, Deep Learning, NLP
    • Robots, drones
  • Since 2015 = 'serious' :: NLP + deep learning
    • & GDE ML; TF&DL co-organiser
    • & Papers...
    • & Dev Course...

About Red Dragon AI

  • Google Partner : Deep Learning Consulting & Prototyping
  • SGInnovate/Govt : Education / Training
  • Products :
    • Conversational Computing
    • Natural Voice Generation - multiple languages
    • Knowledgebase interaction & reasoning

Outline

  • whoami = DONE
  • Problem : Piano Transcription
  • Magenta Project
  • The Model and Losses
  • Code (x2) + Demo
  • Wrap-up

Piano Transcription

Magenta Transcription

MythTV : Piano Spectrum

MythTV Transcription

Added in December 2011...

Magenta Project

Magenta Project Page

Paper Time

Key Result

Transcription F1s

Secret Sauce

  • Pre-processing = mel-spectrogram only
  • Losses are special :
    • Onsets and Frames treated separately
  • Require 'good onset' and 'good hold' for valid output

Model Outputs

Frames and Onsets

Transcription Network

Transcription Model

"Cleaned Up"

Cleaned Frames and Onsets

Demo

  • Notebook should auto-gather code + data
  • But the test audio needs to be found...
  • Have added nicer upload and download interface


https://github.com/
mdda/deep-learning-workshop/
notebooks/work-in-progress/
2018-09_PianoTranscription/
Onsets_and_Frames_mdda.ipynb



Load Directly into Colab

Wrap-up

  • Magenta Project has cool stuff
  • Easy to play around with models
  • TF.slim totally, definitely deprecated
GitHub - mdda

* Please add a star... *

Deep Learning
MeetUp Group

Deep Learning : Jump-Start Workshop

Deep Learning
Developer Course

RedDragon AI
Intern Hunt

  • Opportunity to do Deep Learning all day
  • Work on something cutting-edge
  • Location : Singapore
  • Status : SG/PR FTW
  • Need to coordinate timing...

- QUESTIONS -


Martin @
RedDragon . AI


My blog : http://blog.mdda.net/

GitHub : mdda