By: Andrew Konoff
Read Time: 3 min
AWS has a lot of services, and they all generate logs. A lot of logs. We’ve worked hard to make sure you can capture logs from every source and service on AWS, and today we’re happy to announce the final piece of our AWS logging puzzle: LogDNA’s S3 Collector integration. It’s an easy-to-use Lambda function that lets you ingest any AWS logs that get dumped to S3 – like logs from CloudFront and ELB. Whatever the log, no matter where it comes from on AWS, you can now get it into LogDNA.
If you use Amazon’s various container platforms, like Elastic Kubernetes Service (EKS) and Elastic Container Service (ECS), our LogDNA Agent is there for you. You can also use our LogSpout integration on ECS, if the Agent isn’t convenient or if you’d like a more Docker-focused experience.
But what about all the seeming hundreds of other services that you can use on AWS? Until now, you had to use AWS CloudWatch to aggregate them, and then integrate CloudWatch with LogDNA to send them over to us. That can start to feel like overkill if you’ve just got a few things on AWS, or redundant when you want to keep your log alerting and observation happening in LogDNA.
Introducing LogDNA’s S3 Log Integration
That’s why we’re really happy to offer a lightweight S3 Collection. First, set up your AWS services to dump their logs to S3. Then, you set up our Lambda function, specifying which buckets it should be triggered by. Then anytime new log files get added to those buckets, it’ll send them to LogDNA. It’s easy to set up, highly scalable, and super affordable to run. To get started, just follow the docs here.
We’ve added better parsing of AWS logs too so that your CloudFront and ELB/ALB logs are precisely parsed and searchable. These extremely long ALB logs, for instance, suddenly become a lot easier to understand and query:
For the list of supported parsing types, check out this link. We’re working on adding native parsing for more AWS services, so just let us know what you’d like to see!
Our other AWS Integrations
If you’d like to discover the right way for you to integrate AWS’ many services into LogDNA, just jump into our docs. A few highlights:
Are we missing anything? Let us know by Twitter, or at firstname.lastname@example.org.
The Extract and Aggregate fields feature allows users to custom parse historical logs (post ingestion) and get an aggregated count on those newly parsed fields....
LogDNA is releasing Custom Webhooks, enabling customers to easily integrate LogDNA alerts with 3rd-party services. Custom Webhooks allows you to configure a webhook’s header and...
In the olden days, we used to have to get logs by putting our agent on one machine at a time, like hitching a horse...