Searching for tutorials and software about Deep Learning and Neural Nets? Be sure to look at my Resource Page!
Looking for Octave? Go to my Easy Octave on Mac page!

Tuesday, February 24, 2015

Analysis of the Car experiment.

After an experimental session it is always time for some analysis and introspection.

The net's recognition of the similarities between a  vintage deux-chevaux and a "real car" is impressive. We are seeing AI at work. However, real-world use of the pretrained "PeteNet"  demo, would require some sort of defuzzifier that would reprocess the net output we obtained for the somewhat alien-looking deux-chevaux  into something like (unknown-car-model, 0.75). 

So, as usual, any AI work depends not only on the core AI-engine, but also on how the user interface facilitates the task, and on sophisticated pre and postprocessing of the data. In particular, it is necessary to provide tools to verify and interpret the output and to smoothly handle the surprises which the AI will inevitably dish out. For instance a handwriting recogniser needs not only a good core engine but also a tool to allow the user to correct the wrong guesses while seamlessly continuing with his handwritten input. 

Now for some more fundamental comments. A net such as this one is frozen, and works well with statistically stationary data. But the real world changes with time - new car models appear, apparel fashions evolve etc.  Even presumed constants such as human facial features are not statistically stationary, as these are subject to fashion in makeup and cosmetic surgical enhancement. 

The price of sudden AI failure caused by stale data could be spectacular  - imagine for instance the difficulties caused by an AI car collision avoidance system that is unfamiliar with rollerskaters or skateboarders who suddenly intrude into traffic patterns.

It follows that most AI engines will need to be continually revalidated, and updated, ie. retrained during their work lifetime, and a strategy for doing this will need to be supplied from day one.

Also, backup copies of the engine will need to be retained if the net's present interpretation of data may be needed at a future point in time. Each engine can only see what it is familiar with - so the bottle containing the genie needs a label specifying the vintage.


No comments:

Post a Comment

Hey, let me know what you think of my blog, and what material I should add!