Advanced Natural Language Processing with TensorFlow 2

Build effective real-world NLP applications using NER, RNNs, seq2seq models, Transformers, and more

Ashish Bansal 지음

Packt(GCO Science) 2021년 02월 04일 출간

작품소개

책소개

One-stop solution for NLP practitioners, ML developers, and data scientists to build effective NLP systems that can perform real-world complicated tasks

▶Book Description
Recently, there have been tremendous advances in NLP, and we are now moving from research labs into practical applications. This book comes with a perfect blend of both the theoretical and practical aspects of trending and complex NLP techniques.

The book is focused on innovative applications in the field of NLP, language generation, and dialogue systems. It helps you apply the concepts of pre-processing text using techniques such as tokenization, parts of speech tagging, and lemmatization using popular libraries such as Stanford NLP and SpaCy. You will build Named Entity Recognition (NER) from scratch using Conditional Random Fields and Viterbi Decoding on top of RNNs.

The book covers key emerging areas such as generating text for use in sentence completion and text summarization, bridging images and text by generating captions for images, and managing dialogue aspects of chatbots. You will learn how to apply transfer learning and fine-tuning using TensorFlow 2.

Further, it covers practical techniques that can simplify the labelling of textual data. The book also has a working code that is adaptable to your use cases for each tech piece.

By the end of the book, you will have an advanced knowledge of the tools, techniques and deep learning architecture used to solve complex NLP problems.

▶What You Will Learn
-Grasp important pre-steps in building NLP applications like POS tagging
-Use transfer and weakly supervised learning using libraries like Snorkel
-Do sentiment analysis using BERT
-Apply encoder-decoder NN architectures and beam search for summarizing texts
-Use Transformer models with attention to bring images and text together
-Build apps that generate captions and answer questions about images using custom Transformers
-Use advanced TensorFlow techniques like learning rate annealing,

목차

▶TABLE of CONTENTS
-Chapter 1: Essentials of NLP
-Chapter 2: Understanding Sentiment in Natural Language with BiLSTMs
-Chapter 3: Named Entity Recognition (NER) with BiLSTMs, CRFs, and Viterbi Decoding
-Chapter 4: Transfer Learning with BERT
-Chapter 5: Generating Text with RNNs and GPT-2
-Chapter 6: Text Summarization with Seq2seq Attention and Transformer Networks
-Chapter 7: Multi-Modal Networks and Image Captioning with ResNets and Transformer Networks
-Chapter 8: Weakly Supervised Learning for Classification with Snorkel
-Chapter 9: Building Conversational AI Applications with Deep Learning
-Chapter 10: Installation and Setup Instructions for Code

책 속으로

▶What this book covers
- Chapter 1, Essentials of NLP, provides an overview of various topics in NLP such as tokenization, stemming, lemmatization, POS tagging, vectorization, etc. An overview of common NLP libraries like spaCy, Stanford NLP, and NLTK, with their key capabilities and use cases, will be provided. We will also build a simple classifier for spam.

- Chapter 2, Understanding Sentiment in Natural Language with BiLSTMs, covers the NLU use case of sentiment analysis with an overview of Recurrent Neural Networks (RNNs), LSTMs, and BiLSTMs, which are the basic building blocks of modern NLP models. We will also use tf.data for efficient use of CPUs and GPUs to speed up data pipelines and model training.

- Chapter 3, Named Entity Recognition (NER) with BiLSTMs, CRFs, and Viterbi Decoding, focuses on the key NLU problem of NER, which is a basic building block of taskoriented chatbots. We will build a custom layer for CRFs for improving the accuracy of NER and the Viterbi decoding scheme, which is often applied to a deep model to improve the quality of the output.

- Chapter 4, Transfer Learning with BERT, covers a number of important concepts in modern deep NLP such as types of transfer learning, pre-trained embeddings, an overview of Transformers, and BERT and its application in improving the sentiment analysis task introduced in Chapter 2, Understanding Sentiment in Natural Language with BiLSTMs.

- Chapter 5, Generating Text with RNNs and GPT-2, focuses on generating text with a custom character-based RNN and improving it with Beam Search. We will also cover the GPT-2 architecture and touch upon GPT-3.

- Chapter 6, Text Summarization with Seq2seq Attention and Transformer Networks, takes on the challenging task of abstractive text summarization. BERT and GPT are two halves of the full encoder-decoder model. We put them together to build a seq2seq model for summarizing news articles by generating headlines for them. How ROUGE metrics are used for the evalua

출판사 서평

▶ Preface
2017 was a watershed moment for Natural Language Processing (NLP), with Transformer-and attention-based networks coming to the fore. The past few years have been as transformational for NLP as AlexNet was for computer vision in 2012. Tremendous advances in NLP have been made, and we are now moving from research labs into applications.

These advances span the domains of Natural Language Understanding (NLU), Natural Language Generation (NLG), and Natural Language Interaction (NLI). With so much research in all of these domains, it can be a daunting task to understand the exciting developments in NLP.

This book is focused on cutting-edge applications in the fields of NLP, language generation, and dialog systems. It covers the concepts of pre-processing text using techniques such as tokenization, parts-of-speech (POS) tagging, and lemmatization using popular libraries such as Stanford NLP and spaCy. Named Entity Recognition (NER) models are built from scratch using Bi-directional Long Short-Term Memory networks (BiLSTMs), Conditional Random Fields (CRFs), and Viterbi decoding. Taking a very practical, application-focused perspective, the book covers key emerging areas such as generating text for use in sentence completion and text summarization, multi-modal networks that bridge images and text by generating captions for images, and managing the dialog aspects of chatbots. It covers one of the most important reasons behind recent advances of NLP ？ transfer learning and fine tuning. Unlabeled textual data is easily available but labeling this data is costly. This book covers practical techniques that can simplify the labeling of textual data.

By the end of the book, I hope you will have advanced knowledge of the tools, techniques, and deep learning architectures used to solve complex NLP problems. The book will cover encoder-decoder networks, Long Short-Term Memory networks (LSTMs) and BiLSTMs, CRFs, BERT, GPT-2, GPT-3, Transformers, and other key technologies using TensorFlow.

Advanced TensorFlow techniques required for building advanced models are also covered:

？ Building custom models and layers
？ Building custom loss functions
？ Implementing learning rate annealing
？ Using tf.data for loading data efficiently
？ Checkpointing models to enable long training times (usually several days)

This book contains working code that can be adapted to your own use cases. I hope that you will even be able to do novel state-of-the-art research using the skills you'll gain as you progress through the book.

작가정보

저자(글) Ashish Bansal

Ashish is an AI/ML leader, a well-known speaker, and an astute technologist with over 20 years of experience in the field. He has a Bachelor's in technology from IIT BHU, and an MBA in marketing from Kellogg School of Management. He is currently the Director of Recommendations at Twitch where he works on building scalable recommendation systems across a variety of product surfaces, connecting content to people. He has worked on recommendation systems at multiple organizations, most notably Twitter where he led Trends and Events recommendations and at Capital One where he worked on B2B and B2C products. Ashish was also a co-founder of GALE Partners, a full-service digital agency in Toronto, and spent over 9 years at SapientNitro, a leading digital agency.