Advancements in Natural Language Processing

Have you ever asked Siri for directions, Alexa for a recipe, or Google to find a song title based on a fragment of lyrics? Perhaps you have noticed that the auto-complete on your email or texting app is frighteningly good at predicting what you want to type for your next word, or maybe you have been spooked by seeing an ad for something that you mentioned in passing to a friend in conversation. If you have experienced any of these, it is because of Natural Language Processing.

Natural Language Processing (NLP) is a subset of Artificial Intelligence (AI) that allows computers to understand the structure and meaning of natural human language. NLP works in phases with the two main phases being data preprocessing which involves preparing or cleaning up the text for the computer to be able to read it, and algorithm development which can be either rule based, or machine learning based and allows the computer to understand what it is processing. It is not easy to process human language, especially when we use abstract concepts such as sarcasm and double entendre, or words that have multiple meanings like lead, date, arm, and left.

NLP has applications in everything from IOT devices such as your Echo Dot to data analytics. People generate a lot of text every minute and that text can be processed and used for everything from tailored advertisements to stock trading trends. If you have ever contacted a company for support, you may have experienced NLP without realizing it. NLP is used in AI chat bots and automated phone support to help diagnose issues without the need for a person in a call center. NLP is also used in automatic language transcription and translation, such as with automatic subtitles in YouTube. Researchers use NLP regularly to scrub websites for information and analyze that information for keywords or phrases.

The history of NLP goes back to 1906 when a Swiss linguistics professor by the name of Ferdinand de Saussure developed a series of courses for the University of Geneva that approached language as a system where sounds represent concepts that shift in meaning as the context changes. When he passed away, his colleagues collected his works and released them in a book called “Cours de Linguistique Générale”, which laid the foundation for what we know today as NLP. The technology for NLP began in the 50’s when it was used in the Georgetown–IBM experiment which was performed to demonstrate how computers could be used to translate the Russian language into English. NLP continued to evolve conceptually until the 90s when it’s use in statistics came in to play with the development of machine learning algorithms that were capable of language processing. As computers and artificial intelligence continue to grow and evolve, so has NLP and with recent advancements in neural networks, it has become a very powerful tool for research.

NLP has a bright future and will improve existing technologies as it evolves, bringing us closer to an AI machine being able to pass the Turing test. Some anticipated uses of NLP will include better real-time translation of voice and text, smarter search engines, and advancements in business intelligence. NLP will give structure to unstructured data and provided sentiment analysis to determine the polarity of the data being analyzed (i.e. if it is positive, negative, or neutral). This type of analysis can be used in everything from medical record analysis to predict things like diseases and mental illness to custom advertisement delivery based on things you view or talk about.

Lutkevich, Ben. natural language processing (NLP). techtarget.com. [Online] https://www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP#:~:text=Natural%20language%20processing%20(NLP)%20is,in%20the%20field%20of%20linguistics..
Wikipedia. Wikipedia Georgetown–IBM experiment. Wikipedia. [Online] https://en.wikipedia.org/wiki/Georgetown%E2%80%93IBM_experiment.
Foote, Keith D. A Brief History of Natural Language Processing (NLP). dataversity.net. [Online] https://www.dataversity.net/a-brief-history-of-natural-language-processing-nlp/.
Wikipedia. Natural language processing. Wikipedia. [Online] https://en.wikipedia.org/wiki/Natural_language_processing.
Garbade, Dr. Michael J. A Simple Introduction to Natural Language Processing. becominghuman.ai. [Online] https://becominghuman.ai/a-simple-introduction-to-natural-language-processing-ea66a1747b32.