Thursday, April 4, 2019

Tutorial Nugget: Answering questions about videos with Keras

Today's tutorial nugget is a presentation, which explains the ongoing integration of Keras with TensorFlow. The author of Keras, Fran├žois Chollet details a very nice question answering system about videos, and one can see how Keras seamlessly integrates a pretrained Inception CNN and an LSTM to analyze the videos, and an LSTM processing word embeddings to process the pictures.

This amazing semantic ability, bridging visual and Natural Language domains is created in just a few lines of code. And also, training this really complex architecture becomes feasible for a non-specialist not only because of Google's TF tools, and transfer learning from an Inception net, but also because best practices are integrated into Keras —  think of the all-important LSTM initialization. As Francois puts it, in Keras every default is a good default. 

No comments:

Post a Comment

Hey, let me know what you think of my blog, and what material I should add!