Observability with Grafana

Monitor, control, and visualize your Kubernetes and cloud platforms using the LGTM stack

Rob Chapman , Peter Holmes 지음

Packt(GCO Science) 2024년 01월 12일 출간

작품소개

책소개

Implement the LGTM stack for cost-effective, faster, and secure delivery and management of applications to provide effective infrastructure solutions

▶ Book Description
To overcome application monitoring and observability challenges, Grafana Labs offers a modern, highly scalable, cost-effective Loki, Grafana, Tempo, and Mimir (LGTM) stack along with Prometheus for the collection, visualization, and storage of telemetry data.
Beginning with an overview of observability concepts, this book teaches you how to instrument code and monitor systems in practice using standard protocols and Grafana libraries. As you progress, you’ll create a free Grafana cloud instance and deploy a demo application to a Kubernetes cluster to delve into the implementation of the LGTM stack. You’ll learn how to connect Grafana Cloud to AWS, GCP, and Azure to collect infrastructure data, build interactive dashboards, make use of service level indicators and objectives to produce great alerts, and leverage the AI &ML capabilities to keep your systems healthy. You’ll also explore real user monitoring with Faro and performance monitoring with Pyroscope and k6. Advanced concepts like architecting a Grafana installation, using automation and infrastructure as code tools for DevOps processes, troubleshooting strategies, and best practices to avoid common pitfalls will also be covered.
After reading this book, you’ll be able to use the Grafana stack to deliver amazing operational results for the systems your organization uses.

▶ What You Will Learn
⦁ Understand fundamentals of observability, logs, metrics, and distributed traces
⦁ Find out how to instrument an application using Grafana and OpenTelemetry
⦁ Collect data and monitor cloud, Linux, and Kubernetes platforms
⦁ Build queries and visualizations using LogQL, PromQL, and TraceQL
⦁ Manage incidents and alerts using AI-powered incident management
⦁ Deploy and monitor CI/CD pipelines to automatically validate the desired results
⦁ Take control of observability costs with powerful in-built features
⦁ Architect and manage an observability platform using Grafana

목차

▶ TABLE of CONTENTS
1. Introducing Observability and the Grafana Stack
2. Instrumenting Applications and Infrastructure
3. Setting Up a Learning Environment with Demo Applications
4. Looking at Logs with Grafana Loki
5. Monitoring with Metrics using Grafana Mimir and Prometheus
6. Tracing Technicalities with Grafana Tempo
7. Interrogating Infrastructure with Kubernetes, AWS, GCP, and Azure
8. Displaying Data with Dashboards
9. Managing Incidents using Alerts
10. Automation with Infrastructure as Code
11. Architecting an Observability Platform(N.B. Please use the Look Inside option to see further chapters)

책 속으로

▶ What this book covers
⦁ Chapter 1, Introducing Observability and the Grafana Stack, provides an introduction to the Grafana product stack in relation to observability as a whole. You will learn about the target audiences and how that impacts your design. We will take a look at the roadmap for observability tooling and how Grafana compares to alternative solutions. We will explore architectural deployment models, from self-hosted to cloud offerings. Inevitably, you will have answers to the question “Why choose Grafana?”.
⦁ Chapter 2, Instrumenting Applications and Infrastructure, takes you through the common protocols and best practices for each telemetry type at a high level. You will be introduced to widely used libraries for multiple programming languages that make instrumenting an application simple. Common protocols and strategies for collecting data from infrastructural components will also be discussed. This chapter provides a high-level overview of the technology space and aims to be valuable for quick reference.
⦁ Chapter 3, Setting Up a Learning Environment with Demo Applications, explains how to install and set up a learning environment that will support you through later sections of the book. You will also learn how to explore the telemetry produced by the demo app and add monitoring for your own service.
⦁ Chapter 4, Looking at Logs with, Loki, takes you through working examples to understand LogQL. You will then be introduced to common log formats, and their benefits and drawbacks. Finally, you will be taken through the important architectural designs of Loki, and best practices when working with it.
⦁ Chapter 5, Monitoring with Metrics Using Grafana Mimir and Prometheus, discusses working examples to understand PromQL with real data. Detailed information about the different metric protocols will be discussed. Finally, you will be taken through important architectural designs backing Mimir, Prometheus, and Graphite that guide best practices when working with the tools.
⦁ Chapter 6, Tracing Technicalities with Grafana Tempo, shows you working examples to understand TraceQL with real data. Detailed information about the different tracing protocols will be discussed. Finally, you will be taken through the important architectural designs of Tempo, and best practices when working with it.
⦁ Chapter 7, Interrogating Infrastructure with Kubernetes, AWS, GCP, and Azure, describes the setup and configuration used to capture telemetry from infrastructure. You will learn about the different options available for Kubernetes. Additionally, you will investigate the main plugins that allow Grafana to query data from cloud vendors such as AWS, GCP, and Azure. You will look at solutions to handle large volumes of telemetry where direct connections are not scalable. The chapter will also cover options for filtering and selecting telemetry data before it gets to Grafana for security and cost optimization.
⦁ Chapter 8, Displaying Data with Dashboards, explains how you can set up your first dashboard in the Grafana UI. You will also learn how to present your telemetry data in an effective and meaningful way. The chapter will also teach you how to manage your Grafana dashboards to be organized and secure.
⦁ Chapter 9, Managing Incidents Using Alerts, describes how to set up your first Grafana alert with Alert Manager. You will learn how to design an alert strategy that prioritizes business-critical alerts over ordinary notifications. Additionally, you will learn about alert notification policies, different delivery methods, and what to look for.
⦁ Chapter 10, Automation with Infrastructure as Code, gives you the tools and approaches to automate parts of your Grafana stack deployments while introducing standards and quality checks. You will gain a deep dive into the Grafana API, working with Terraform, and how to protect changes with validation.
⦁ Chapter 11, Architecting an Observability Platform, will show those of you who are responsible for offering an efficient and easy-to-use observability platform how you can structure your platform so you can delight your internal customers. In the environment we operate in, it is vital to offer these platform services as quickly and efficiently as possible, so more time can be dedicated to the production of customer-facing products. This chapter aims to build on the ideas already covered to get you up and running quickly.
⦁ Chapter 12, Real User Monitoring with Grafana, introduces you to frontend application observability, using Grafana Faro and Grafana Cloud Frontend Observability for real user monitoring (RUM). This chapter will discuss instrumenting your frontend browser applications. You will learn how to capture frontend telemetry and link this with backend telemetry for full stack observability.
⦁ Chapter 13, Application Performance with Grafana Pyroscope and k6, introduces you to application performance and profiling using Grafana Pyroscope and k6. You will obtain a high-level overview that discusses the various aspects of k6 for smoke, spike, stress, and soak tests, as well as using Pyroscope to continuously profile an application both in production and test environments.
⦁ Chapter 14, Supporting DevOps Processes with Observability, takes you through DevOps processes and how they can be supercharged with observability with Grafana. You will learn how the Grafana stack can be used in the development stages to speed up the feedback loop for engineers. You will understand how to prepare engineers to operate the product in production. Finally, You will learn when and how to implement CLI and automation tools to enhance the development workflow.
⦁ Chapter 15, Troubleshooting, Implementing Best Practices, and More with Grafana, closes the book by taking you through best practices when working with Grafana in production. You will also learn some valuable troubleshooting tips to support you with high-traffic systems in day-to-day operations. You will also learn about additional considerations for your telemetry data with security and business intelligence.

출판사 서평

▶ Preface
Hello and welcome! Observability with Grafana is a book about the tools offered by Grafana Labs for observability and monitoring. Grafana Labs is an industry-leading provider of open source tools to collect, store, and visualize data collected from IT systems. This book is primarily aimed toward IT engineers who will interact with these systems, whatever discipline they work in. We have written this book as we have seen some common problems across organizations:
• Systems that were designed without a strategy for scaling are being pushed to handle additional data load or teams using the system
• Operational costs are not being attributable correctly in the organization, leading to poor cost analysis and management
• Incident management processes that treat the humans involved as robots without sleep schedules or parasympathetic nervous systems
In this book, we will use the OpenTelemetry Demo application to simulate a real-world environment and send the collected data to a free Grafana Cloud account that we will create. This will guide you through the Grafana tools for collecting telemetry and also give you hands-on experience using the administration and support tools offered by Grafana. This approach will teach you how to run the Grafana tools in a way so that anyone can experiment and learn independently.
This is an exciting time for Grafana, identified as a visionary in the 2023 Gartner Magic Quadrant for Observability (https://www.gartner.com/en/documents/4500499). They recently delivered change in two trending areas:
• Cost reduction: This has seen Grafana as the first vendor in the observability space to release tools that not only help you understand your costs but also reduce them.
• Artificial intelligence (AI): Grafana has introduced generative AI tools that assist daily operations in simple yet effective ways – for example, writing an incident summary automatically. Grafana Labs also recently purchased Asserts.ai to simplify root cause analysis and accelerate problem detection.
We hope you enjoy learning some new things with us and have fun doing it!

작가정보

저자(글) Rob Chapman

Rob Chapman is a creative IT engineer and founder at The Melt Cafe, with two decades of experience in the full application life cycle. Working over the years for companies such as the Environment Agency, BT Global Services, Microsoft, and Grafana, Rob has built a wealth of experience on large complex systems. More than anything, Rob loves saving energy, time, and money and has a track record for bringing production-related concerns forward so that they are addressed earlier in the development cycle, when they are cheaper and easier to solve. In his spare time, Rob is a Scout leader, and he enjoys hiking, climbing, and, most of all, spending time with his family and six children.

저자(글) Peter Holmes

Peter Holmes is a senior engineer with a deep interest in digital systems and how to use them to solve problems. With over 16 years of experience, he has worked in various roles in operations. Working at organizations such as Boots UK, Fujitsu Services, Anaplan, Thomson Reuters, and the NHS, he has experience in complex transformational projects, site reliability engineering, platform engineering, and leadership. Peter has a history of taking time to understand the customer and ensuring Day-2+ operations are as smooth and cost-effective as possible.