Building Fast High-Performance Recognition Systems with Recurrent Neural Networks and LSTM Thomas Breuel, Volkmar Frinken, Marcus Liwicki In this half-day tutorial several Recurrent Neural Networks (RNNs) and their application to Pattern Recognition will be described. First, a brief history of RNNs is presented. Next, several problems of simple RNNs are described and the Long Short-Term Memory (LSTM) is presented as a solution for those problems. For a better understanding of the network, its behaviour on several toy problems and real-world PR-applications is investigated. Finally, extended architectures, such as the bi- and multi- directional LSTM will be proposed and their application to speech, handwriting and other PR-domains will be given. Existing Open-Source Toolkits implementing the LSTM and some extensions will be presented and an introduction of how to use these tools will be given.

• RNNLIB - the original C++ library implementing LSTM and many of the ideas about LSTM
• JANNlab - Java-based implementation of 1D and BLSTM, no CTC
• OCRopus - Python-based implementation of 1D and BLSTM, with CTC (the implementation is in lstm.py; here is an example of using lstm.py).