Home > Program > Tutorials > AM-06
ICPR2012 Tutorials AM-06
Long Short-Term Memory Networks for Pattern Recognition
Lecturer:
Abstract: In this half-day tutorial several Recurrent Neural Networks (RNNs) and their application to Pattern Recognition will be described. First, a brief history of RNNs is presented. Next, several problems of simple RNNs are described and the Long Short-Term Memory (LSTM) is presented as a solution for those problems. For a better understanding of the network, its behaviour on several toy problems and real-world PR-applications is investigated. Finally, extended architectures, such as the bi- and multi- directional LSTM will be proposed and their application to speech, handwriting and other PR-domains will be given. Existing Open-Source Toolkits implementing the LSTM and some extensions will be presented and an introduction of how to use these tools will be given. |
Course description
Relevant References:
- Y. Bengio, P. Simard, and P. Frasconi. Learning Long-Term Dependencies with Gradient Descent is Difficult, IEEE Transactions on Neural Networks, VOL. 5, NO. 2, MARCH 1994
- S. Hochreiter, J. Schmidhuber. LSTM can Solve Hard Long Time Lag Problems, NIPS'9, 1997
- S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. IEEE Press, 2001.
- Alex Graves, Jürgen Schmidhuber. Framewise phoneme classification with bidirectional LSTM and other neural network architectures, Neural Networks, 2005
- Graves, A. and Liwicki, M. and Fernandez, S. and Bertolami, R. and Bunke, H. and Schmidhuber, J.: A Novel Connectionist System for Unconstrained Handwriting Recognition, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 31, No. 5 (2009) 855-868.
- Book: Alex Graves: Supervised Sequence Labelling with Recurrent Neural Networks (Studies in Computational Intelligence), Springer Berlin Heidelberg, 2012, ISBN-13: 978-3642247965
About Lecturer:
Marcus Liwicki received his M.S. degree in Computer Science from the Free University of Berlin, Germany, in 2004, and his PhD degree from the University of Bern, Switzerland, in 2007. Subsequently, he successfully finished his Habiliation and received the postdoctoral lecture qualification from the Technical University of Kaiserslautern, Germany, in 2011. Currently he is a senior researcher and private lecturer at the German Research Center for Artificial Intelligence (DFKI). His research interests include knowledge management, semantic desktop, electronic pen-input devices, on-line and off-line handwriting recognition and document analysis. From October 2009 to March 2010 he visited Kyushu University (Fukuoka, Japan) as a research fellow, supported by the Japanese Society for the Promotion of Science.
Marcus Liwicki is a member of the IAPR and a regular reviewer for international journals, including IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Audio, Speech and Language Processing, International Journal of Document Analysis and Recognition, Pattern Recognition, and Pattern Recognition Letters. He is a board member of the International Graphonomics Society and a PC-Chair of the International Workshop on Automated Forensic Handwriting Analysis. Furthermore he serves in the program committee of the Int. Conference on Frontiers in Handwriting Recognition, the International Conference on Document Analysis and Recognition, the Int. Workshop on Document Analysis Systems, the Workshop on Analytics for Noisy Unstructured Text Data, Document Recognition and Retrieval Conference, the IEEE ISM Workshop on Multimedia Technologies for E-Learning, the Conference of the International Graphonomics Society, and as a reviewer of several IAPR conferences, AI workshops and conferences. Since 2010 he is the organizer of the discussion groups on the Workshops on Document Analysis and Recognition. Marcus Liwicki gave a number of invited talks at several international workshops, universities, and companies. He also gave several tutorials on IAPR conferences. Marcus Liwicki is a co-author of the book "Recognition of Whiteboard Notes – Online, Offline, and Combination", published by World Scientific in October 2008. He has more than 80 publications, including more than ten journal papers.
Archive: Call for Tutorial Proposals
