Doing OCR in Javascript

Name Card OCR v0.01

Martin Andrews / @redcatlabs

25 August 2014

Motivation

  • MeetUp Junkie
  • Lots of Name Cards
    • Stacked in neat piles
    • Occasionally scanned
    • Follow-ups ?
  • Itch-to-be-Scratched

Alternatives

  • Existing phone apps : "Phone Home"
  • JS OCR (Emscripten) on HN
    • Nice technology demo
    • OCR not great (ocrad.js)
    • Better OCR ~> server-side = Tesseract
  • Linked-In / Meetup / CRM

OCR for Name Cards

  • OCR works on books fairly well
  • Cards are rather "free form"
  • Few words => Little context

What Cards Look Like

"As Good as can be Expected"

Good Image

What Cards Look Like

Webcam

Webcam

What Cards Look Like

Confusing

Confusing

What Cards Look Like

Obvious to Humans, but...

Webcam

Not so obvious
for machines...

Is the top the same brightness as the side?

Illusion

( try covering up the edge in the middle )

Tools Used

  • image processing : dv
  • procmail : mailparser
  • smtp : emailjs nodemailer
  • webserver : expressjs (+ bootstrap)

Interactive Demo

  • PLEASE send an image of a Business Card to :
    • ocr@redcatlabs.com
Fill Required

TIPS : No flash; Fill middle third;
Landscape; Dark background

Demo : Terms of Service (v0.01)

  • Won't sell your email to anyone else
  • Won't sell your card photos to anyone else
  • Won't put your card photos up on-screen
  • Send you one response now
  • Send you another response soon, once the OCR thing works better
  • Hopefully, there'll be an update at a Talk.js night

Incoming Email

procmail.js


## ~/.procmailrc  (0600)
:0
* ^TOocr*
| node ~/SUBDIRECTORY/procmail.js >> ~/Mail/ocr.log
   
## procmail.js   
var MailParser = require("mailparser").MailParser;
var mailparser = new MailParser();

mailparser.on("end", function(mail_object) {
  console.log("From:", mail_object.from);       // [{address:'sender@example.com',name:'Sender Name'}]
  for(var i=0; i<mail_object.attachments.length; i++) {
    var attachment = mail_object.attachments[i];
    console.log(attachment.fileName, attachment.length);
  }
});
 

On GitHub : https://github.com/mdda/procmail.js

Image Processing

dv = leptonica + tesseract


var dv = require('dv');
image = new dv.Image('jpg', fs.readFileSync(image_filename));

var grey = image.toGray();

var box = {
  x: grey.width/3, width:grey.width/3, 
  y: grey.height/3, height:grey.height/3
};
var cropped = grey.crop(box);
var histogram = cropped.histogram();

// ...

var tess = new dv.Tesseract('eng', grey);
tess.pageSegMode='sparse_text';
console.log(tess.findText('plain'));
 

Outgoing Email

smtp : emailjs nodemailer


var nodemailer = require('nodemailer');
var smtpTransport = require('nodemailer-smtp-transport');

var transporter = nodemailer.createTransport(smtpTransport({
  host: 'localhost', port: 25,
//  auth: { user: 'username', pass: 'password' },
  maxConnections: 5, maxMessages: 10
}));

transporter.sendMail({
   from:    "OCR <OCR@outgoing.example.com>", 
   to:      "YOU <you@example.com>",
   subject: "Test from Nodemailer",
   text:    "Body Text"
}, function(err, message) { console.log(err || message); });
 

Web Server

webserver : expressjs (+ bootstrap + jade) + "forever"

Super-easy to use a template from GitHub

Niggles with living in subdirectory


# github.com/primaryobjects/Node.js-Bootstrap-Starter-Template

app.use('/ocr', express.static(path.join(__dirname, 'public')));
var subdir = '/ocr';
app.get(subdir+'/', routes.index);

## nginx :
location ~ ^/ocr/(img|fonts|js|css)/ {
  root   /FULL-PATH-TO/webserver/; 
  rewrite ^/ocr/(.+)$ /public/$1 break;    
} 
location /ocr/ {
  proxy_pass http://localhost:3000; # Leave off trailing '/'
}
 

Future Work

  • OCR : even the basics are not-so-easy
    • Name Cards are special case
    • Large variety, few clues
    • The more examples available, the better
  • Want to do this 'on the phone'
    • Local processing = less server load
    • Local processing = more privacy
    • Looking at deep learning .js modules
  • Try it yourself ! :: ocr@RedCatLabs.com

Questions ?

 

Martin.Andrews @
RedCatLabs.com

 

 

Have a Go at ::

ocr @ RedCatLabs.com