NEW: RESOURCE PAGE

Searching for tutorials and software about Deep Learning and Neural Nets? Be sure to look at my Resource Page!
Looking for Octave? Go to my Easy Octave on Mac page!

Tuesday, February 24, 2015

A Shallow Net exercise with OpenCV

Disclaimer — if you find my writing here stilted, please remember that these are basically working notes. Reporting my activities here means I have a backup record to locate all this stuff again. I must be the world's most sloppy scientist. And the lack of originality is intended. I am only publishing my reading and exercises.

Today we will improvise a bit, and use OpenCV  to run some 3-layer Shallow Net Python code. You may wonder why we would use OpenCV for a non-graphical experiment. The answer is that the OpenCV library comes with a predefined set of primitives for coding neural net models.

I came across Matt Zucker's Python code for an experiment that trains a net to recognise the sort of bar-numbers you would find on an old style 7-segment LED calculator display. I know this example well, as I published a paper about it about 20 years ago. entitled "Feature induction by backpropagation", but at the time I had to write my own simulator to run the model for reasons you will understand if you read the abstract at the bottom of this post. Today with the help of a library, this is a small hack.





The first job we have is bringing up OpenCV.  Luckily there's already a script to load OpenCV into a Vagrant box available on Git. In my case I just ported the Zip file across to my Mac from Git, unpacked it into a directory to find a Vagrantfile and a Bootstrap file, and ran "Vagrant Up". It took a long time to build OpenCV and SciPy. I guess I should try and post that VM Box somewhere for people to use.

I then copied Matt Zucker's code to the VM's Mac  directory, and it appeared in /vagrant on the VM.

Then I ran the code! Hooray The experiment was a success!  As we can see from the output below, Matt's code recognises digits 0-9.

I now have a working Python framework with which to write simple neural net code, which I can easily use while I'm struggling with the complexities of Caffe  and friends.


vagrant@ppy-opencv:~/simple$ python simple-digits.py
libdc1394 error: Failed to initialize libdc1394
ran for 1005 iterations
inputs:
[[ 1.  1.  1.  1.  0.  1.  1.  0.  1.  1.  0.  1.  1.  1.  1.]
 [ 1.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  1.  1.  1.]
 [ 1.  1.  1.  0.  0.  1.  1.  1.  1.  1.  0.  0.  1.  1.  1.]
 [ 1.  1.  1.  0.  0.  1.  1.  1.  1.  0.  0.  1.  1.  1.  1.]
 [ 1.  0.  1.  1.  0.  1.  1.  1.  1.  0.  0.  1.  0.  0.  1.]
 [ 1.  1.  1.  1.  0.  0.  1.  1.  1.  0.  0.  1.  1.  1.  1.]
 [ 1.  1.  1.  1.  0.  0.  1.  1.  1.  1.  0.  1.  1.  1.  1.]
 [ 1.  1.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.  0.  0.  1.]
 [ 1.  1.  1.  1.  0.  1.  1.  1.  1.  1.  0.  1.  1.  1.  1.]
 [ 1.  1.  1.  1.  0.  1.  1.  1.  1.  0.  0.  1.  1.  1.  1.]]
targets:
[[ 1. -1. -1. -1. -1. -1. -1. -1. -1. -1.]
 [-1.  1. -1. -1. -1. -1. -1. -1. -1. -1.]
 [-1. -1.  1. -1. -1. -1. -1. -1. -1. -1.]
 [-1. -1. -1.  1. -1. -1. -1. -1. -1. -1.]
 [-1. -1. -1. -1.  1. -1. -1. -1. -1. -1.]
 [-1. -1. -1. -1. -1.  1. -1. -1. -1. -1.]
 [-1. -1. -1. -1. -1. -1.  1. -1. -1. -1.]
 [-1. -1. -1. -1. -1. -1. -1.  1. -1. -1.]
 [-1. -1. -1. -1. -1. -1. -1. -1.  1. -1.]
 [-1. -1. -1. -1. -1. -1. -1. -1. -1.  1.]]
predictions:
[[ 0.87214456 -1.0798411  -1.66724626 -0.92884208 -0.98613688 -1.00556657
  -1.01725002 -1.00168128 -0.94426154 -1.00826581]
 [-1.14249101  0.96739166 -1.1555237  -0.98833244 -0.99838906 -1.02076795
  -0.94675515 -1.00213278 -1.02079769 -0.96274348]
 [-1.7125664  -1.06722585  0.86052547 -0.9514627  -0.99274433 -0.98942425
  -0.99719121 -1.00148912 -0.92819438 -1.01527162]
 [-0.89838177 -0.93892495 -0.83733626  0.97307133 -1.00415977 -1.04926685
  -1.04014692 -0.99467156 -1.14358189 -0.96448026]
 [-0.96628756 -0.99527804 -0.95907845 -1.00227794  1.0005437  -1.00898408
  -1.00837635 -1.00097141 -1.01140273 -0.9972425 ]
 [-0.96207846 -1.02954467 -0.97951868 -1.00741557 -1.00686177  0.78750484
  -0.82156218 -1.01160181 -1.58137503 -0.87867077]
 [-0.89221623 -0.93540225 -0.8748513  -1.00847621 -1.00423206 -0.79187145
   0.79065247 -0.9817474  -0.80952586 -1.65647678]
 [-0.97360473 -0.98527592 -0.9836571  -1.01752647 -1.00335966 -0.99554984
  -0.99132159  0.99921353 -1.00417662 -0.99869791]
 [-0.92637606 -0.95399084 -0.91519803 -1.10987311 -1.00394962 -1.59982816
  -0.80860406 -1.01443547  0.74639276 -0.86932467]
 [-0.98752284 -0.98742131 -1.01109933 -0.9754243  -1.00532989 -0.78630859
  -1.60476965 -0.98549547 -0.7444202   0.8465155 ]]
sum sq. err: 3.16442744112
accuracy: 1.0
vagrant@ppy-opencv:~/simple$ Connection to 127.0.0.1 closed by remote host.



NB. Cited:
Ronald, Edmund, Marc Schoenauer, and Martine Sebag. "Feature induction by backpropagation." Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on. Vol. 1. IEEE.
Abstract:
A method for investigating the internal knowledge representation constructed by neural net learning is described: it is shown how from a given weight matrix defining a feedforward artificial neural net, we can induce characteristic patterns of each of the classes of inputs classified by that net. These characteristic patterns, called prototypes, are found by a gradient descent search of the space of inputs. After an exposition of the theory, results are given for the well known LED recognition problem where a network simulates recognition of decimal digits displayed on a seven-segment LED display. Contrary to theoretical intuition, the experimental results indicate that the computed prototypes retain only some of the features of the original input patterns. Thus it appears that the indicated method extracts those features deemed significant by the net

No comments:

Post a Comment

Hey, let me know what you think of my blog, and what material I should add!