ICDAR 2013 Tutorial

Thanks to everybody coming to the tutorial and letting us share our experiences and excitement about LSTM and recurrent neural networks. We hope you've found it useful.

Building Fast High-Performance Recognition Systems with Recurrent Neural Networks and LSTM
Thomas Breuel, Volkmar Frinken, Marcus Liwicki

In this half-day tutorial several Recurrent Neural Networks (RNNs) and their application to Pattern Recognition will be described. First, a brief history of RNNs is presented. Next, several problems of simple RNNs are described and the Long Short-Term Memory (LSTM) is presented as a solution for those problems. For a better understanding of the network, its behaviour on several toy problems and real-world PR-applications is investigated. Finally, extended architectures, such as the bi- and multi- directional LSTM will be proposed and their application to speech, handwriting and other PR-domains will be given. Existing Open-Source Toolkits implementing the LSTM and some extensions will be presented and an introduction of how to use these tools will be given.


For the tutorial slides, please go to the Files section

Recommended mplementations:
  • RNNLIB - the original C++ library implementing LSTM and many of the ideas about LSTM
  • JANNlab - Java-based implementation of 1D and BLSTM, no CTC
  • OCRopus - Python-based implementation of 1D and BLSTM, with CTC (the implementation is in lstm.py; here is an example of using lstm.py).
For other implementations mentioned in the tutorial, please contact us.