AWS CloudWatch Logs gives you full visibility into your AWS infrastructure, from individual workloads to the services that bind them. Monitoring these logs helps ensure their smooth and continued operation, ongoing stability, and performance. Integrating CloudWatch Logs with LogDNA makes it easier to parse, search, and analyze AWS logs in order to detect anomalies and troubleshoot problems faster.
LogDNA integrates with CloudWatch using a Lambda function, which streams logs directly from CloudWatch to LogDNA’s ingestion servers. We recently updated this function with a new runtime and greater customizability. In this article, we’ll show you how to update your existing LogDNA integration to the latest version.
This article assumes you have already set up CloudWatch integration with LogDNA. If not, please refer to the AWS CloudWatch Logs documentation.
Changes Introduced in the New Version
For the most part, the new integration behaves exactly the same as the old version. Logs are still piped from CloudWatch to the function, then forwarded to LogDNA’s ingestion servers. The biggest change is a switch from Python to Node.js.
We’ve introduced a retry mechanism to the integration to improve the delivery of logs. We’ve also change the structure of the log being forwarded so that fields like group and stream are now indexed and searchable outside of the meta field.
This version also supports additional configuration options, including:
- Maximum lengths for each log line
- Timeout period for requests
- Keepalive options for open TCP sockets
- Maximum number of retries in case of a network failure
- How long to wait before each retry
As with the old version, you can still specify a custom hostname, tags, and ingestion URL.
Updating the Integration
Performing the update is simply a matter of replacing your existing Lambda function’s code and changing the runtime. You retain all of your original settings including your log groups and environment variable, and the new function is deployed with no downtime.
Start by downloading the latest ZIP file containing the function’s code. Open the Lambda Console and select the function that you want to update. Click on the function in the Designer view to display the Function Code panel. Change the code entry type to “Upload a .zip file”, and select “Node.js 10.x” (or higher) as the runtime. Make sure the Handler textbox displays “index.handler”.
You can add or confirm any environment variables by scrolling down to the Environment Variables panel. At a minimum, you should have a LOGDNA_KEY variable with your LogDNA Ingestion key as its value. If you want to test the function, you can do so by following these instructions. When you are done, click Save to build and deploy the function. Now, any new logs received by CloudWatch will be picked up by the new function and streamed to LogDNA.
There are a number of benefits from streaming logs from CloudWatch to LogDNA, including automatic parsing, super-fast searching, live tail, alerts, charts, and more. Whether you’re hosting a relational database, Kubernetes cluster, or any other resource, LogDNA can help you monitor your AWS infrastructure much more easily and much faster.