Mastering Reinforcement Learning with Python

Build next-generation, self-learning models using reinforcement learning techniques and best practices

Enes Bilgin 지음

Packt(GCO Science) 2020년 12월 18일 출간

작품소개

책소개

Get hands-on experience in creating state-of-the-art reinforcement learning agents using TensorFlow and RLlib to solve complex real-world business and industry problems with the help of expert tips and best practices

▶What You Will Learn
-Model and solve complex sequential decision-making problems using RL
-Develop a solid understanding of how state-of-the-art RL methods work
-Use Python and TensorFlow to code RL algorithms from scratch
-Parallelize and scale up your RL implementations using Ray's RLlib package
-Get in-depth knowledge of a wide variety of RL topics
-Understand the trade-offs between different RL approaches
-Discover and address the challenges of implementing RL in the real world

▶Key Features
-Understand how large-scale state-of-the-art RL algorithms and approaches work
-Apply RL to solve complex problems in marketing, robotics, supply chain, finance, cybersecurity, and more
-Explore tips and best practices from experts that will enable you to overcome real-world RL challenges

▶Who This Book Is For
This book is for expert machine learning practitioners and researchers looking to focus on hands-on reinforcement learning with Python by implementing advanced deep reinforcement learning concepts in real-world projects. Reinforcement learning experts who want to advance their knowledge to tackle large-scale and complex sequential decision-making problems will also find this book useful. Working knowledge of Python programming and deep learning along with prior experience in reinforcement learning is required.

목차

▶TABLE of CONTENTS
▷Section 1: Reinforcement Learning Foundations
Chapter 1: Introduction to Reinforcement Learning
Chapter 2: Multi-Armed Bandits
Chapter 3: Contextual Bandits
Chapter 4: Makings of a Markov Decision Process
Chapter 5: Solving the Reinforcement Learning Problem

▷Section 2: Deep Reinforcement Learning
Chapter 6: Deep Q-Learning at Scale
Chapter 7: Policy-Based Methods
Chapter 8: Model-Based Methods
Chapter 9: Multi-Agent Reinforcement Learning

▷Section 3: Advanced Topics in RL
Chapter 10: Introducing Machine Teaching
Chapter 11: Achieving Generalization and Overcoming Partial Observability
Chapter 12: Meta-Reinforcement Learning
Chapter 13: Exploring Advanced Topics

▷Section 4: Applications of RL
Chapter 14: Solving Robot Learning
Chapter 15: Supply Chain Management
Chapter 16: Personalization, Marketing, and Finance
Chapter 17: Smart City and Cybersecurity
Chapter 18: Challenges and Future Directions in Reinforcement Learning

책 속으로

▶What this book covers
- Chapter 1, Introduction to Reinforcement Learning, provides an introduction to RL, presents motivating examples and success stories, and looks at RL applications in industry. It then gives some fundamental definitions to refresh your mind on RL concepts and concludes with a section on software and hardware setup.

- Chapter 2, Multi-Armed Bandits, covers a rather simple RL setting, bandit problems without context, which, on the other hand, has tremendous applications in industry as an alternative to the traditional A/B testing. The chapter also describes a very fundamental RL trade-off: exploration versus exploitation. It then presents three approaches to tackle this trade-off and compares them against A/B testing.

- Chapter 3, Contextual Bandits, takes the discussion on multi-armed bandits to an advanced level by adding context to the decision-making process and involving deep neural networks in decision making. We adapt a real dataset from the U.S. Census to an online advertising problem. We conclude the chapter with a section on the applications of bandit problems in industry and business.

- Chapter 4, Makings of a Markov Decision Process, builds the mathematical theory behind sequential decision processes that are solved using RL. We start with Markov chains, where we describe types of states, ergodicity, transitionary, and steady-state behavior. Then we go into Markov reward and decision processes. Along the way, we introduce return, discount, policy, value functions, and Bellman optimality, which are key concepts in RL theory that will be frequently referred to in later chapters. We conclude the chapter with a discussion on partially observed Markov decision processes. Throughout the chapter, we use a grid world example to illustrate the concepts.

- Chapter 5, Solving the Reinforcement Learning Problem, presents and compares dynamic programming, Monte Carlo, and temporal-difference methods, which are fundamental to understanding how

출판사 서평

▶ Preface
Reinforcement learning (RL) is a field of artificial intelligence (AI) used for creating self-learning autonomous agents. Building on a strong theoretical foundation, this book takes a practical approach and uses examples inspired by real-world industry problems to teach you about state-of-the-art RL.

Starting with bandit problems, Markov decision processes, and dynamic programming, the book provides an in-depth review of the classical RL techniques, such as Monte Carlo methods and temporal-difference learning. After that, you will learn about deep Q-learning, policy gradient algorithms, actor-critic methods, model-based methods, and multi-agent reinforcement learning. Then, you'll be introduced to some of the key approaches behind the most successful RL implementations, such as domain randomization and curiosity-driven learning.

As you advance, you'll explore many novel algorithms with advanced implementations using modern Python libraries such as TensorFlow and Ray's RLlib package. You'll also find out how to implement RL in areas such as robotics, supply chain management, marketing, finance, smart cities, and cybersecurity while assessing the trade-offs between different approaches and avoiding common pitfalls.

By the end of this book, you'll have mastered how to train and deploy your own RL agents for solving RL problems.

인물정보

저자(글) Enes Bilgin

Enes Bilgin works as a senior AI engineer and a tech lead in Microsoft's Autonomous Systems division. He is a machine learning and operations research practitioner and researcher with experience in building production systems and models for top tech companies using Python, TensorFlow, and Ray/RLlib. He holds an M.S. and a Ph.D. in systems engineering from Boston University and a B.S. in industrial engineering from Bilkent University. In the past, he has worked as a research scientist at Amazon and as an operations research scientist at AMD. He also held adjunct faculty positions at the McCombs School of Business at the University of Texas at Austin and at the Ingram School of Engineering at Texas State University.