NEW: RESOURCE PAGE

Searching for tutorials and software about Deep Learning and Neural Nets? Be sure to look at my Resource Page!
Looking for Octave? Go to my Easy Octave on Mac page!

Thursday, April 9, 2015

Continuing with FANN on MNIST

Looking at Andrea's FANN for MNIST work is interesting, recompiling his code has given me a good running start for some experiments I want to do. Here are some notes on these various tools, and a code review of Andrea's works.

UPDATE: AFTER A COUPLE OF WEEKS I CONCLUDED THAT ANDREA'S CODE ORGANISATION IS USEFUL, BUT HIS CODE IS REALLY BUGGY, AND I REWROTE A LOT OF IT.  FANN ON THE OTHER HAND SEEMS SOLID. 

  1. Character data can be found on Yann LeCun's MNIST page. However, it is known that -as should be- the training set and the validation set are not by the same writers. So, is the validation data really equivalent in character (pun intended)  to  the training data? If not, the pattern recognition problem we think we're solving has in fact been turned into an AI problem. 
  2. Andrea's software basically consists of a set of small single-use C++ programs, namely convert, train, test and bulk-test. The decomposition into the sequence convert-train-test is nicely conceived. Each of these reads some intermediate data-files according to the Unix philosophy of small programs linked by files rather than big monolithic software. 
  3. Convert reads in the MNIST pixel grid grey-scale data and labels and decodes them into a file readable by FANN.
  4. Train reads in the data file generated by convert and runs a number of trianing epochs. 
  5. Test checks the net classification of a randomly chosen specific exemplar input against the MNIST data label. 
  6. Bulk-test runs a statistical test on numerous examples chosen at random.
  7. To create each step of this software, Andrea just put some user-friendly command-line argument decoding wrappers around the core FANN routines, and set some default string values that point them at default data file names. As far as I can see the only use of C++ primitives is for the IO, the rest is standard C.
  8. As Andrea is obviously a much better programmer than I ever will be myself, his code is a good starting point for evolving my own FANN wrappers. And yes, I did spot  at least one bug. 
  9. FANN is really really fast. On my laptop, I can do about 1 training epoch per second through the complete MNIST data with incremental backprop, yielding a mediocre trained classifier  in a minute or so. 
Here is an example of what one can expect. Note that US "7" is written in a way very similar to a french "1". 

Edmunds-MBP:src edmundronald$ ./test network.nn

Network tester
==============

+----------------------------+
|                            |
|                            |
|                            |
|                            |
|                            |
|                            |
|                            |
|                 ..XX..     |
|              ..XXXXXX.     |
|          ...XXXXXXXX.      |
|      ...XXXXXXX..XXX       |
|     .XXXXXXXX.. .XX.       |
|     .XXXX..    .XXX        |
|      ..        .XX.        |
|               .XXX.        |
|              .XXX.         |
|              .XX.          |
|              XXX.          |
|             .XXX.          |
|            .XXX.           |
|            XXX.            |
|           .XXX.            |
|           .XX.             |
|          .XX.              |
|        ...XX.              |
|        .XXX.               |
|         .XX.               |
|                            |
+----------------------------+
P(0) = 3.06362e-15; 
P(1) = 1.48933e-12; 
P(2) = 1.77737e-17; 
P(3) = 3.51025e-11; 
P(4) = 0.00300184; 
P(5) = 3.55816e-13; 
P(6) = 3.14016e-07; 
P(7) = 1; 
P(8) = 1.635e-08; 
P(9) = 1.28438e-09; 


  *** CORRECT ***

Example number 39053
Expected  : 7
Calculated: 7


PS.---------
Links for anyone wanting to look at the MNIST data files etc. 
http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html Hinton's m code for training, backprop and autoencoder
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3644707/ cuda mnist and also m code for reading files
https://chrisjmccormick.wordpress.com/2015/01/10/understanding-the-deeplearntoolbox-cnn-example/
http://martin-thoma.com/classify-mnist-with-pybrain/



1 comment:

  1. Hello, I'm looking to test FANN against other networks (such as tensor flow), is this code available somewhere for testing?

    ReplyDelete

Hey, let me know what you think of my blog, and what material I should add!