Hands-On Vision and Behavior for Self-Driving Cars

Explore visual perception, lane detection, and object classification with Python 3 and OpenCV 4

Luca Venturi , Krishtof Korda 지음

Packt(GCO Science) 2020년 10월 23일 출간

작품소개

책소개

A practical guide to learning visual perception for self-driving cars for computer vision and autonomous system engineers

▶Book Description
The visual perception capabilities of a self-driving car are powered by computer vision. The work relating to self-driving cars can be broadly classified into three components - robotics, computer vision, and machine learning. This book provides existing computer vision engineers and developers with the unique opportunity to be associated with this booming field.

You will learn about computer vision, deep learning, and depth perception applied to driverless cars. The book provides a structured and thorough introduction, as making a real self-driving car is a huge cross-functional effort. As you progress, you will cover relevant cases with working code, before going on to understand how to use OpenCV, TensorFlow and Keras to analyze video streaming from car cameras. Later, you will learn how to interpret and make the most of lidars (light detection and ranging) to identify obstacles and localize your position. You'll even be able to tackle core challenges in self-driving cars such as finding lanes, detecting pedestrian and crossing lights, performing semantic segmentation, and writing a PID controller.

By the end of this book, you'll be equipped with the skills you need to write code for a self-driving car running in a driverless car simulator, and be able to tackle various challenges faced by autonomous car engineers.

▶What You Will Learn
？ Understand how to perform camera calibration
？ Become well-versed with how lane detection works in self-driving cars using OpenCV
？ Explore behavioral cloning by self-driving in a video-game simulator
？ Get to grips with using lidars
？ Discover how to configure the controls for autonomous vehicles
？ Use object detection and semantic segmentation to locate lanes, cars, and pedestrians
？ Write a PID controller to control a self-driving car running in a simulator

▶Key Features
？Explore the building blocks of the visual perception system in self-driving cars
？Identify objects and lanes to define the boundary of driving surfaces using open-source tools like OpenCV and Python
？Improve the object detection and classification capabilities of systems with the help of neural networks

▶Who This Book Is For
This book is for software engineers who are interested in learning about technologies that drive the autonomous car revolution. Although basic knowledge of computer vision and Python programming is required, prior knowledge of advanced deep learning and how to use sensors (lidar) is not needed.

목차

▶TABLE of CONTENTS
▷ Section 1: OpenCV and Sensors and Signals
？ Chapter 1: OpenCV Basics and Camera Calibration
？ Chapter 2: Understanding and Working with Signals
？ Chapter 3: Lane Detection

▷ Section 2: Improving How the Self-Driving Car Works with Deep Learning and Neural Networks
？ Chapter 4: Deep Learning with Neural Networks
？ Chapter 5: Deep Learning Workflow
？ Chapter 6: Improving Your Neural Network
？ Chapter 7: Detecting Pedestrians and Traffic Lights
？ Chapter 8: Behavioral Cloning
？ Chapter 9: Semantic Segmentation

▷ Section 3: Mapping and Controls
？ Chapter 10: Steering, Throttle, and Brake Control
？ Chapter 11: Mapping Our Environments

책 속으로

▶What this book covers
？ Chapter 1, OpenCV Basics and Camera Calibration, is an introduction to OpenCV and NumPy; you will learn how to manipulate images and videos, and how to detect pedestrians using OpenCV; in addition, it explains how a camera works and how OpenCV can be used to calibrate it.

？ Chapter 2, Understanding and Working with Signals, describes the different types of signals: serial, parallel, digital, analog, single-ended, and differential, and explains some very important protocols: CAN, Ethernet, TCP, and UDP.

？ Chapter 3, Lane Detection, teaches you everything you need to know to detect the lanes in a road using OpenCV. It covers color spaces, perspective correction, edge detection, histograms, the sliding window technique, and the filtering required to get the best detection.

？ Chapter 4, Deep Learning with Neural Networks, is a practical introduction to neural networks, designed to quickly teach how to write a neural network. It describes neural networks in general and convolutional neural networks in particular. It introduces Keras, a deep learning module, and it shows how to use it to detect handwritten digits and to classify some images.

？ Chapter 5, Deep Learning Workflow, ideally complements Chapter 4, Deep Learning with Neural Networks, as it describes the theory of neural networks and the steps required in a typical workflow: obtaining or creating a dataset, splitting it into training, validation, and test sets, data augmentation, the main layers used in a classifier, and how to train, do inference, and retrain. The chapter also covers underfitting and overfitting and explains how to visualize the activations of the convolutional layers.

？ Chapter 6, Improving Your Neural Network, explains how to optimize a neural network, reducing its parameters, and how to improve its accuracy using batch normalization, early stopping, data augmentation, and dropout.

？ Chapter 7, Detecting Pedestrians and Traffic Lights, introduces you to CARLA, a self-driving car simulator, which we will use to create a dataset of traffic lights. Using a pre-trained neural network called SSD, we will detect pedestrians, cars, and traffic lights, and we will use a powerful technique called transfer learning to train a neural network to classify the traffic lights according to their colors.

？ Chapter 8, Behavioral Cloning, explains how to train a neural network to drive CARLA. It explains what behavioral cloning is, how to build a driving dataset using CARLA, how to create a network that's suitable for this task, and how to train it. We will use saliency maps to get an understanding of what the network is learning, and we will integrate it with CARLA to help it self-drive!

？ Chapter 9, Semantic Segmentation, is the final and most advanced chapter about deep learning, and it explains what semantic segmentation is. It details an extremely interesting architecture called DenseNet, and it shows how to adapt it to semantic segmentation.

？ Chapter 10, Steering, Throttle, and Brake Control, is about controlling a self-driving car. It explains what a controller is, focusing on PID controllers and covering the basics of MPC controllers. Finally, we will implement a PID controller in CARLA.

？ Chapter 11, Mapping Our Environments, is the final chapter. It discusses maps, localization, and lidar, and it describes some open source mapping tools. You will learn what Simultaneous Localization and Mapping (SLAM) is and how to implement it using the Ouster lidar and Google Cartographer.

출판사 서평

▶ Preface
Self-driving cars will soon be among us. The improvements seen in this field have been nothing short of extraordinary. The first time I heard about self-driving cars, it was in 2010, when I tried one in the Toyota showroom in Tokyo. The ride cost around a dollar. The car was going very slowly, and it was apparently dependent on sensors embedded in the road.

Fast forward a few years, lidar and advancements in computer vision and deep learning have made that technology look primitive and unnecessarily invasive and expensive.

In the course of this book, we will use OpenCV for a variety of tasks, including pedestrian detection and lane detection; you will discover deep learning and learn how to leverage it for image classification, object detection, and semantic segmentation, using it to identify pedestrians, cars, roads, sidewalks, and crossing lights, while learning about some of the most influential neural networks.

You will get comfortable using the CARLA simulator, which you will use to control a car using behavioral cloning and a PID controller; you will learn about network protocols, sensors, cameras, and how to use lidar to map the world around you and to find your position.

But before diving into these amazing technologies, please take a moment and try to imagine the future in 20 years. What are the cars like? They can drive by themselves. But can they also fly? Are there still crossing lights? How fast, heavy, and expensive are those cars? How do we use them, and how often? What about self-driving buses and trucks?

We cannot know the future, but it is conceivable that self-driving cars, and self-driving things in general, will shape our daily lives and our cities in new and exciting ways.

Do you want to play an active role in defining this future? If so, keep reading. This book can be the first step of your journey.

작가정보

저자(글) Luca Venturi

Luca Venturi has extensive experience as a programmer with world-class companies, including Ferrari and Opera Software. He has also worked for some start-ups, including Activetainment (maker of the world's first smart bike), Futurehome (a provider of smart home solutions), and CompanyBook (whose offerings apply artificial intelligence to sales). He worked on the Data Platform team at Tapad (Telenor Group), making petabytes of data accessible to the rest of the company, and is now the lead engineer of Piano Software's analytical database.

저자(글) Krishtof Korda

Krishtof Korda grew up in a mountainside home over which the US Navy's Blue Angels flew during the Reno Air Races each year. A graduate from the University of Southern California and the USMC Officer Candidate School, he set the Marine Corps obstacle course record of 51 seconds. He took his love of aviation to the USAF, flying aboard the C-5M Super Galaxy as a flight test engineer for 5 years, and engineered installations of airborne experiments for the USAF Test Pilot School for 4 years. Later, he transitioned to designing sensor integrations for autonomous cars at Lyft Level 5. Now he works as an applications engineer for Ouster, integrating LIDAR sensors in the fields of robotics, AVs, drones, and mining, and loves racing Enduro mountain bikes.