Disclaimer: I published this article first on TechBeacon, you can read the original article here.
Microservices are solving many architectural problems that have plagued developers and operations for years. But while the benefits are now clear, many professionals aren’t fully aware of the many challenges that come with microservices. A significant portion of those center around monitoring and observability.
Microservices add complexity to your system, communication, and network, which makes monitoring more complicated. Thankfully, you can find lots of tutorials, books, tools, and case studies from organizations that have successfully implemented effective monitoring and observability in a microservices environment.
Here is our list of the most popular and useful tools and resources shared within the software engineering, site-reliability engineer (SRE), and DevOps communities.
Tutorials and guides for monitoring microservices
A software engineer from Hootsuite explains how service relationships, dependencies, and status are paramount to keeping his team, the build pipeline, and the application running smoothly. This talk includes a section on the API that’s baked into Hootsuite’s service-oriented architecture, enabling them to monitor and explore application service dependency graphs in real time.
James Turnbull is the author of The Art of Monitoring. This conversation with him provides a good overview of the state of microservices monitoring.
This roundup talks to engineers from IBM, the Financial Times, and Netflix about the pain points of microservices monitoring, then moves on to describe each firm’s best practices for maintaining and monitoring microservices.
This guide explains the challenges of building distributed architectures and the required changes to scale the monitoring of microservices. It emphasizes the ways in which various system architectures can influence how you respond to incidents and resolve them.
Despite the benefits of using a microservices-based architecture, they add a lot of “noise.” Monitoring a microservices-based environment without taking this noise into account can be very confusing. Here’s how you can eliminate at least some of it.
This post explains the relationship between containers (Docker containers in this example) and microservices. It also covers the advantages of using these technologies and the challenges of monitoring Docker. “What should you look for in a Docker monitoring solution?” Three things, the author says.
This article explains how containerization and microservices changed the way developers write and run their applications. The author predicts what new monitoring and instrumentation practices will be utilized in 2018.
This blog post offers up a real-world example of a monitoring architecture for microservices that uses a variety of open-source tools, including Graphana, Collectd, Heapster, and Brubeck.
This excerpt from the book Production-Ready Microservices, by Susan Fowler, provides a list of useful questions to assess the production-readiness of microservices and a microservice ecosystem. The questions are related to key metrics, logging, dashboards, alerting, and on-call rotations to give you an idea of what kinds of tools and processes you need to have in place to effectively monitor microservices.
This pitfalls-style post explains the performance, fault tolerance, and monitoring challenges when running microservices. Then it gives the reader some tools and suggestions to overcome those challenges.
This detailed guide on Node.js monitoring with the Prometheus monitoring framework will help you understand Node.js-based microservices monitoring by covering code instrumentation (and its risks), data collection, data visualization, queries, alerting, and Kubernetes integration.
This tutorial shows you how to set up monitoring in a cloud-native microservices environment using Prometheus Operator to monitor Envoy proxy. As the author explains, one reason for Envoy’s growing popularity is its emphasis on observability. Instead of using Envoy directly, the tutorial uses Ambassador.
You can use another proxy, Zipkin, with Kubernetes (and Google Container Engine) to send trace information when monitoring microservices. Learn how it works in this article.
Networking knowledge is critical in microservices monitoring. Here’s why.
Here are several considerations for deploying a Docker-based microservices architecture — and some best practices for externalizing, monitoring, and managing microservices.
Monitoring & observability tools
This article offers up a great list of tools for monitoring and debugging microservices.
Prometheus is an open-source service-monitoring system and time-series database.
Spigo is a tool that simulates protocol interactions in the Go language. You can use it to run microservices architecture simulations and generate new visualizations and metrics.
This daemon collects system and application performance metrics over time, and provides mechanisms to store the values in a variety of ways. It gathers metrics from many sources, including the operating system, applications, log files, and external devices. It can then store this information or make it available over the network.
A tool for monitoring distributed systems, Riemann can forward any event stream to Graphite, a popular monitoring framework and time-series database. It also provides phone, SMS, and email alerts.
Use Heapster to monitor Kubernetes container clusters. It collects and interprets various signals such as compute resource usage and lifecycle events, supporting multiple sources of data.
This container-monitoring tool, developed by Google, natively supports Docker containers. The cAdvisor daemon, which collects, aggregates, processes, and exports information about containers, is available in a graphical version.
Sysdig is a monitoring and troubleshooting tool that captures system calls and events from the Linux kernel, and lets you to save, filter, and analyze the data from the command line or from within a desktop app. The open-source developers of Sysdig describe it as “strace + tcpdump + htop + iftop + lsof + wireshark for your entire system.”
A metrics dashboard, Grafana integrates with many data sources, including Graphite, CloudWatch, Prometheus, Elsaticsearch, and InfluxDB.
The Trace visualized distributed tracing platform is designed for Node.js microservices.
Sensu helps developers monitor servers, services, application health, and business KPIs. It collects and analyzes custom metrics and can send failure notifications. You can use Sensu with containers such as Docker, RKT, and LXC.
Are there any other resources that you find particularly useful in the field of microservices monitoring and observability? Share them in the comments below.