By: Thu Nguyen
Read Time: 6 min
I hosted a webinar where I covered why logging is important, how to choose a logging provider. And then shared our experience of setting up logging on Kubernetes containers, the Kubernetes logging framework and the logging best practices we’ve implemented internally and supported our customers who run Kubernetes in production.
LogDNA’s ultimate goal is to provide a useful tool for developers to be able to quickly access logs, to get in, get your logs and get back to work. Our UI, UX, speed of search and simplicity in integration is built to solve this one problem. Our Kubernetes, or container logging integration is a prime example of this, with just two kubectl lines, your container logs are sent to LogDNA. There’s no configuration, no messy sidecar container that consumes additional resources, dependencies on fluentd.
The logs are ephemeral: when containers output logs to stdout, the containers automatically dumps the files to /var/log. When the pods are evicted, crashed, deleted or scheduled on a different node, the logs from the container is gone. This is different than logging on traditional servers or virtual machines. When your apps die on your virtual machine, you don’t lose the logs until you delete it. In Kubernetes, it cleans up after itself and the logs will not persist. Understanding the ephemeral nature of default logging on Kubernetes is important because it points to a centralized log management solution.
When we first started using Kubernetes, we, like many others started with Fluentd. You can find many guides for setup, even though there might be 30+ steps. It is easy to get started, there are plenty of examples of configs online. Fluentd works well in low volume but the challenge is with higher volume. Scaling becomes challenging, mainly the efforts of scaling ElasticSearch, learning how to properly architect the shards and indices and becoming an expert of ElasticSearch operations.
Similar to the fluentd agent, you can install the LogDNA agent to read and stream the log lines from /var/log. Since the agent streams the logs to LogDNA, it doesn’t take additional resources in the pods and nodes. You can then focus on scaling your product instead of spending your time scaling the logging infrastructure. The agent runs per node instead of per pod which is much more efficient.
We have kept things simple with our kubernetes integration and all you need to do is install our agent using two kubectl commands
We’ll automatically extract and index the Kubernetes metadata (pod name, container name, namespace, node) and auto parse all the common log types without the need to manually identify them like you would using fluentd. You can also set up your custom log lines to be parsed and indexed by us using our custom log parsing interface.
Try our Kubernetes integration today with a 14-day free trial. We look forward to hearing your feedback.
Isn’t all logging pretty much the same? Logs appear by default, like magic, without any further intervention by teams other than simply starting a system…...
First published on www.ibm.com on October 7, 2019. Written by: Norman Hsieh, VP of Business Development, LogDNA You know what they say: you can’t fix what you can’t...
First published as a case study on www.ibm.com on October 3, 2019. What is Log Analysis? IBM Cloud™ Log Analysis with LogDNA enables you to quickly find...