Deep Learning in Natural Language Processing
DOI:
https://doi.org/10.5281/zenodo.15303476Keywords:
Deep Learning, Natural Language Processing, Discriminative Models, Generative Models, Hybrid Models, Machine LearningAbstract
Recent breakthroughs in machine learning have provided the artificial intelligence field with the necessary capabilities to address persistent challenges. This paper provides a brief overview of deep learning, classified into three main categories: discriminative, generative, and hybrid models. Natural language processing (NLP) is one of the domains that has considerably profited from these breakthroughs. Deep learning techniques have allowed the efficient handling of many complex tasks related to natural language processing. This paper emphasizes important NLP tasks and notable projects that have used deep learning to tackle them. Despite the wide range of NLP tasks, the results show that deep learning continuously surpasses conventional methods across numerous applications.
References
A. Ekbal and S. Bandyopadhyay, "Part of speech tagging in bengali using support vector machine," Information Technology, 2008. ICIT08. International Conference on. IEEE, 2008, pp. 106-111.
A. Ekbal, S. Mondal, and S. Bandyopadhyay, "Pos tagging using hmm and rule-based chunking," The Proceedings of SPSAL, 2007, pp. 25-28.
A. Hassan and A. Mahmood, "Deep Learning approach for sentiment analysis of short texts," International Conference on Control, Automation and Robotics (ICCAR), 3rd, 2017, pp. 705-710.
A. Krizhevsky, I. Sutskever and G. Hinton, "ImageNet classification with deep convolutional neural Networks," Advances in neural information processing systems, 2012, pp. 1097-1105.
A. M. Mohsen, N. M. El-Makky and N. Ghanem, "Author Identification Using Deep Learning," IEEE International Conference on Machine Learning and Applications (ICMLA), 15th, 2016, pp. 898-903.
A. Paul, B. Purkayastha and S. Sarkar, "Hidden Markov Model based Part of Speech Tagging for Nepali language," International Symposium on Advanced Computing and Communication (ISACC), 2015, pp. 149- 156.
A. Saxena,"Convolutional Neural Networks: An Illustrated Explanation," Crossroads the ACM magazine for students, http://xrds.acm.org/blog/2016/06/convolutional-neural-networks-cnns-illustrated-explanation/, 2016, accessed on May 2017.
A. Y. Ng and M. I. Jordan On Discriminative vs. Generative classifiers: A comparison of logistic regression and nave Bayes," Advances in neural information processing systems, vol. 2, 2002, pp. 841-848.
C. Qian, T. He and R. Zhang, "Deep Learning based Authorship Identification,".
E. Kumar "Natural Language Processing," I. K. International Pvt Ltd, 2011.
G. G. Chowdhury, S. Osindero and Y. Teh, "Natural language Processing," Annual review of information science and technology, vol. 37, no. 1, 2003, pp. 51-89.
G. Hinton, S. Osindero and Y. Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, 2006, pp. 1527-1554.
H. Ney, "Speech translation: Coupling of recognition and translation," Proc. ICASSP, 1999.
J. Dean et al., "Large Scale Distributed Deep Networks," Advances in neural information processing systems, 2012, pp. 1223-1231.
J. Xu, H. Li and S. Zhou, "An Overview of Deep Generative Models," IETE Technical Review vol. 32, no. 2, 2014, pp. 131-139.
L. Deng "Three classes of deep learning architectures and their applications: a tutorial survey," APSIPA transactions on signal and information processing, 2012.
L. Deng and N. Jaitly, Deep discriminative and generative models for pattern recognition," USENIX-Advanced Computing Systems Association, 2015.
L. Deng and X. Li "Machine learning paradigms in speech recognition: An overview," IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 5, 2013, pp. 1060-1089.
M. F. Kabir, K. Abdullah-Al-Mamun and M. N. Huda, "Deep learning based parts of speech tagger for Bengali," International Conference on Informatics, Electronics and Vision (ICIEV), 2016, pp. 26-29.
M. Nielsen, "Neural Networks and Deep Learning," 2017.
P. Vateekul and T. Koomsubha, "A study of sentiment analysis using deep learning techniques on Thai Twitter data," International Joint Conference on Computer Science and Software Engineering (JCSSE), 13th, 2016.
R. S. Dudhabaware and M. S. Madankar, "Review on Natural Language Processing Tasks for Text Documents," IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), 2014.
R. Sarikaya, G. Hinton and A. Deoras, "Application of deep belief networks for natural language understanding," IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), vol. 22, no. 4, 2014, pp. 778-784.
R. Sarikaya, G. Hinton and A. Deoras, "Application of deep belief networks for natural language understanding," IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), vol. 22, no. 4, 2014, pp. 778-784.
R. Vijayakrishnan et al., "Prevalence of Heart Failure Signs and Symptoms in a Large Primary Care Population Identified Through the Use of Text and Data Mining of the Electronic Health Record," Journal of cardiac failure, 2014.
S. Sun, H. Liu and H. Lin, "Twitter part-of-speech tagging using pre-classification Hidden Markov model," IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2012, pp. 1118-1123.
T. Duyu, B. Qin and T. Liu, "Document modeling with gated recurrent neural network for sentiment classication," In Proceedings of EMNLP, 2015, pp. 1422-1432.
T. Sainath, A. Mohamed, B. Kingsbury and B. Ramabhadran, "Convolutional neural networks for LVCSR," IEEE International Conference on ICASSP, 2013, pp. 8614-8618.
X. He and L. Deng "Speech recognition, machine translation, and speech translation — A unifying discriminative framework," IEEE Sig. Proc. Magazine, vol. 28, 2011.
X. Wang, J. Zhang and Y. Yan, "Support Vector Machine for Chinese Part-Of-Speech Tagging in SpeechSynthesis Systems," International Con- ference on Biomedical Engineering and Computer Science, 2010.
X. Zhang, J. Zhao and Y. LeCun, "Character-level convolutional networks for text classification," Advances in neural information processing systems, 2015, pp. 649-657.
X. Zheng, H. Chen, and T. Xu, Deep learning for chinese word segmentation and pos tagging," EMNLP, 2013, pp. 647-657.
X. Zhou, J. Guo and R. Bie, "Deep Learning Based Affective Model for Speech Emotion Recognition," Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communi- cations, Cloud and Big Data Computing, Internet of People, and Smart World Congress (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), 2016 Intl IEEE Conferences, 2016, pp. 841-846.
Y. LeCun et al., Gradient-based learning applied to document recognition," Proceedings of the IEEE vol. 86, no. 11, 1998, pp. 2278-2324.
Y. LeCun, Y. Bengio and G. Hinton, "Deep learning," Nature vol. 521, 2014, pp. 436-444.
Y. N. Dauphin, A. Fan, D. Grangier and M. uli, "Language modeling with gated convolutional networks," arXiv preprint arXiv:1612.0808, 2016.
Y. Tsuboi, "Neural networks leverage corpus-wide information for part-of-speech tagging," Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 938-950.
Y. Xiang, Q. Chen, X. Wang and Y. Qin, "Answer Selection in Community Question Answering via Attentive Neural Networks," IEEE Signal Processing Letters, vol. 24, no. 4, 2017, pp. 505-509.
Z. w. Huang, W. t. Xue, and Q. r. Mao, peech emotion recognition with unsupervised feature learning," Frontiers of Information Technology & Electronic Engineering, vol. 16, 2015, pp. 358-366.
Downloads
Published
How to Cite
Issue
Section
License
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
