Optical character recognition with neural networks
DOI:
https://doi.org/10.26577/JMMCS-2018-4-572Keywords:
OCR, neural network, convolutional neural networksAbstract
XXI century is the age of global automation and digitization. There is high demand for optical recognition software, including character recognition. There are different approaches in solution optical recognition problem. Some of them based on classical feature extraction methods. Other based on machine learning algorithms. In this work, we observed related works in machine learning field and propose the plan for further research. The work relies on two research studies that describe basics and fundamentals of machine learning. These researches include various experiments in this field. We tried to repeat these experiments to get acquainted with methods and techniques and to identify key features that are affecting on optical character recognition process. We analyzed two main architectural structures: multilayer perceptron and convolutional neural network. In conclusion, we learned key points of machine learning techniques and composed our own strategy for further researches. Further work will cover researches and experiments on performance of several architectures. In addition, we observed latest tools, software programs and environments for the most convenient way to organize implementation process
References
1. Yann LeCun, Yoshua Bengio and Geoffrey Hinton, ''Deep learning'', Nature (2015): 436–444.
2. Behnam Neyshabur et al., ''Exploring Generalization in Deep Learning'', accessed October 14, 2018,https://papers.nips.cc/paper/7176-exploring-generalization-in-deep-learning.pdf.
3. Warren S. McCulloch and Walter H. Pitts, ''A logical calculus of the ideas immanent in nervous activity'', Bulletin of Mathematical Biophysics (Springer US, 1943): 115–133.
4. Vidushi Sharma, Sachin Rai and Anurag Dev, ''A Comprehensive Study of Artificial Neural Networks'', International Journal of Advanced Research in Computer Science and Software Engineering 10 (2012): 278-284.
5. Jayesh B. Ahire, ''Real world Applications of Artificial Neural Networks'', accessed October 14, 2018, https://medium.com/@jayeshbahire/real-world-applications-of-artificial-neural-networks-a6a6bc17ad6a
6. Sumit Das et al., ''Applications of Artificial Intelligence in Machine Learning: Review and Prospect'', International Journal of Computer Applications 115 (2015): 31-41.
7. Sonali B. Maind and Wankar Priyanka, ''Research Paper on Basic of Artificial Neural Network'', International Journal on Recent and Innovation Trends in Computing and Communication 2 (2014), accessed November 5, 2018, http://www.ijritcc.org/download/Research\%20Paper\%20on\%20Basic\%20of\%20Artificial\%20Neural\%20Network.pdf.
8. Alex Krizhevsky, Ilya Sutskever and Geoffery E. Hinton, ''Imagenet classification with deep convolutional neural networks'', Advances in neural information processing systems 25 (2012), accessed November 5, 2018, https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.
9. Christian Szegedy et al., ''Rethinking the inception architecture for computer vision'', Paper presented at the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 26 – July 1, 2016.
10. Michael Nielsen, ''Neural network and Deep learning'', accessed October 20, 2018, http://neuralnetworksanddeeplearning.com/index.html.
11. Yann LeCun et al., ''Gradient-based learning applied to document recognition'', Proceedings of the IEEE 86 (1998):2278-2324.
12. Vishnu Sundaresan and Jasper Lin, ''Recognizing Handwritten Digits and Characters'', accessed October 20, 2018, http://cs231n.stanford.edu/reports/2015/pdfs/vishnu\_final.pdf.
13. Michael Nielsen, ''Neural Network and Deep Learning: Learning with gradient descent'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
14. Michael Nielsen, ''Neural Network and Deep Learning: The architecture of neural networks'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
15. Michael Nielsen, ''Neural Network and Deep Learning: Perceptron'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
16. Michael Nielsen, ''Neural Network and Deep Learning: Sigmoid function'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
17. Michael Nielsen, ''Neural Network and Deep Learning: Implementing our network to classify digits'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
18. Michael Nielsen, ''Neural Network and Deep Learning: A simple network to classify handwritten digits'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
19. Saleh Albelwi and Ausif Mahmood, ''A Framework for Designing the Architectures of Deep Convolutional Neural Networks'', Entropy 19(2017): 242-262.
20. Amit Choudhary, ''A Review of Various Character Segmentation Techniques for Cursive Handwritten Words Recognition", International Journal of Information \& Computation Technology 4 (2014): 559-564.
21. Shuang Wu et al., ''L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks", IEEE Transactions on Neural Networks and Learning Systems (2018), accessed November 28, 2018, doi:10.1109/TNNLS.2018.2876179.
22. Sergey Ioffe and Christian Szegedy, ''Batch normalization: Accelerating deep network training by reducing internal covariate shift'', Paper presented at the 32nd International Conference on Machine Learning, Lille, France, July 06 – 11, 2015.
23. Yann LeCun et al., ''Efficient backprop, Neural Networks: Tricks of the Trade'', second edition (Springer US, 1998): 9-48.
24. Yann LeCun et al., ''A handwritten digit recognition: Applications of neural net chips and automatic learning'', Neurocomputing (Springer US, 2005): 303-318.
25. Alex Krizhevsky, Ilya Sutskever and Geoffery E. Hinton, ''ImageNet Classification with Deep Convolutional Neural Networks'', accessed November 20, 2018, https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
2. Behnam Neyshabur et al., ''Exploring Generalization in Deep Learning'', accessed October 14, 2018,https://papers.nips.cc/paper/7176-exploring-generalization-in-deep-learning.pdf.
3. Warren S. McCulloch and Walter H. Pitts, ''A logical calculus of the ideas immanent in nervous activity'', Bulletin of Mathematical Biophysics (Springer US, 1943): 115–133.
4. Vidushi Sharma, Sachin Rai and Anurag Dev, ''A Comprehensive Study of Artificial Neural Networks'', International Journal of Advanced Research in Computer Science and Software Engineering 10 (2012): 278-284.
5. Jayesh B. Ahire, ''Real world Applications of Artificial Neural Networks'', accessed October 14, 2018, https://medium.com/@jayeshbahire/real-world-applications-of-artificial-neural-networks-a6a6bc17ad6a
6. Sumit Das et al., ''Applications of Artificial Intelligence in Machine Learning: Review and Prospect'', International Journal of Computer Applications 115 (2015): 31-41.
7. Sonali B. Maind and Wankar Priyanka, ''Research Paper on Basic of Artificial Neural Network'', International Journal on Recent and Innovation Trends in Computing and Communication 2 (2014), accessed November 5, 2018, http://www.ijritcc.org/download/Research\%20Paper\%20on\%20Basic\%20of\%20Artificial\%20Neural\%20Network.pdf.
8. Alex Krizhevsky, Ilya Sutskever and Geoffery E. Hinton, ''Imagenet classification with deep convolutional neural networks'', Advances in neural information processing systems 25 (2012), accessed November 5, 2018, https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.
9. Christian Szegedy et al., ''Rethinking the inception architecture for computer vision'', Paper presented at the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 26 – July 1, 2016.
10. Michael Nielsen, ''Neural network and Deep learning'', accessed October 20, 2018, http://neuralnetworksanddeeplearning.com/index.html.
11. Yann LeCun et al., ''Gradient-based learning applied to document recognition'', Proceedings of the IEEE 86 (1998):2278-2324.
12. Vishnu Sundaresan and Jasper Lin, ''Recognizing Handwritten Digits and Characters'', accessed October 20, 2018, http://cs231n.stanford.edu/reports/2015/pdfs/vishnu\_final.pdf.
13. Michael Nielsen, ''Neural Network and Deep Learning: Learning with gradient descent'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
14. Michael Nielsen, ''Neural Network and Deep Learning: The architecture of neural networks'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
15. Michael Nielsen, ''Neural Network and Deep Learning: Perceptron'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
16. Michael Nielsen, ''Neural Network and Deep Learning: Sigmoid function'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
17. Michael Nielsen, ''Neural Network and Deep Learning: Implementing our network to classify digits'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
18. Michael Nielsen, ''Neural Network and Deep Learning: A simple network to classify handwritten digits'', accessed October 21, 2018, http://neuralnetworksanddeeplearning.com/chap1.html.
19. Saleh Albelwi and Ausif Mahmood, ''A Framework for Designing the Architectures of Deep Convolutional Neural Networks'', Entropy 19(2017): 242-262.
20. Amit Choudhary, ''A Review of Various Character Segmentation Techniques for Cursive Handwritten Words Recognition", International Journal of Information \& Computation Technology 4 (2014): 559-564.
21. Shuang Wu et al., ''L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks", IEEE Transactions on Neural Networks and Learning Systems (2018), accessed November 28, 2018, doi:10.1109/TNNLS.2018.2876179.
22. Sergey Ioffe and Christian Szegedy, ''Batch normalization: Accelerating deep network training by reducing internal covariate shift'', Paper presented at the 32nd International Conference on Machine Learning, Lille, France, July 06 – 11, 2015.
23. Yann LeCun et al., ''Efficient backprop, Neural Networks: Tricks of the Trade'', second edition (Springer US, 1998): 9-48.
24. Yann LeCun et al., ''A handwritten digit recognition: Applications of neural net chips and automatic learning'', Neurocomputing (Springer US, 2005): 303-318.
25. Alex Krizhevsky, Ilya Sutskever and Geoffery E. Hinton, ''ImageNet Classification with Deep Convolutional Neural Networks'', accessed November 20, 2018, https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Downloads
How to Cite
Shalakhmetov, A., & Aubakirov, S. (2019). Optical character recognition with neural networks. Journal of Mathematics, Mechanics and Computer Science, 100(4), 28–41. https://doi.org/10.26577/JMMCS-2018-4-572
Issue
Section
Computer Science
