Since its first release in 2007, Splunk quickly became one of the leading log management solutions. Its focus on enterprise grade log analysis and security incident and event management (SIEM) made it the de facto choice for organizations generating large volumes of log files and machine data. But over the past decade, the log management landscape has changed drastically. Modern distributed architectures such as microservices, containers, and hybrid clouds are the new norm, and organizations have new options that are better optimized for managing their log data.
What does Splunk do?
Splunk is more than just a log collection tool. It’s extremely expensive because it’s feature-rich for enterprise level organizations. The Splunk tool ingests, parses, and indexes all kinds of machine data including event logs, server logs, files, and network events. This data can be used to monitor activity and issues in your infrastructure, look for trends in operational performance, trigger alerts after detecting unusual behavior, and correlating events. In addition to logging, Splunk is a big data analytics platform and SIEM solution.
Why Splunk isn’t the best log management system
Despite being a feature-rich platform, there are many drawbacks to using Splunk. In addition to cost (up to $4,500+ per GB) and their unfavorable payment model, developers also complain about the slow search speed, inability to handle large amounts of data, the complexity of its setup process, outdated user interface, and the need for onboarding and special training.
The good thing is that with many more log management tools available, there are plenty of alternatives available. To ensure that we’re comparing apples to apples, we made a list of features and requirements to look for in your new logging system.
6 logging features to look for in the right alternative
- Ingest data in a number of different formats from different sources
- Parse and index log data into fields, allowing for searching and filtering
- Real-time (or near real-time) log monitoring with the ability to set alerts
- Visualizations using search screens, dashboards, graphs, and charts
- Capable of running at enterprise scale
- Cheaper than Splunk
- Bonus feature – the ability to deploy anywhere – cloud, self-hosted, on-premises, or multi-cloud
Based on these features, here’s our list of Splunk competitors to help you choose the absolute best alternative in the log management and analysis space for your business.
5 Best Splunk Alternatives
LogDNA was created to solve many of the key challenges present in other log management solutions. With deployment models available for cloud-based, on-premise, private cloud, and hybrid/multi-cloud, LogDNA offers a large degree of flexibility for organizations ranging from small businesses to enterprises.
Log collection is extremely simple. You can collect logs from hosts using an installed agent, or send logs directly from applications or platforms such as AWS, Docker, Kubernetes, Heroku, and syslog.
LogDNA is built for speed and accessibility. Built on a super-optimized Elasticsearch, LogDNA lets you index, filter, and tail logs instantaneously. The web-based UI is built to be straightforward and intuitive, allowing you to quickly filter by key fields and group logs by source. In addition to supporting custom views and graphs, the LogDNA web UI gives you the ability to create custom dashboards or provide user-specific event logs to customers.
Unlike many log management solutions, LogDNA is priced by usage with no data caps. You only pay for what you use. Plans start at $1.50 per GB per month, which includes unlimited ingestion and a week of retention. Enterprise plans start at just $3 per GB per month for up to 30 days of retention (and significantly longer for HIPAA compliance). LogDNA offers a fully featured free 14-day trial to get started.
2. Elastic Stack
The Elastic Stack (previously the ELK stack) has the distinction of being an open source log management solution. It actually consists of four separate projects:
- Elasticsearch, a search and analytics engine
- Logstash, a log ingestion, and processing pipeline
- Kibana, a data visualization tool for Elasticsearch
- Beats, a set of agents that collect and send data to Logstash
The base installation provides all of the tools needed to ship, ingest, and view log data using a web-based UI. Because it’s open source, users can download and run the Elastic Stack for free. This also means the Elastic Stack benefits from an active developer community, hundreds of plugins, and support for a diverse array of input formats and sources.
However, running the Elastic Stack is not as straightforward as other solutions. As a primarily self-hosted solution, the Elastic Stack needs extensive setup and configuration before it can work as an enterprise scale log management solution. Although Elastic—the company that maintains the Elastic Stack—offers cloud-hosted Elasticsearch as a service, hosted Logstash and hosted Kibana services are only available through third-party providers such as AWS and Azure.
In addition, the free version of the Elastic Stack is limited in its functionality. Features common to other log management solutions such as access controls, alerting, reporting, and graphing are only available through a subscription. The Elastic Stack is also expensive to host, costing nearly $2,000,000 to run at enterprise scale over a period of just three years.
Fluentd is a tool for ingesting structured, unstructured, and semi-structured data sets. It acts as an intermediary between data sources and outputs, allowing it to convert and route data for a number of different platforms, services, applications, and programming languages. As an open source tool, Fluentd is used as a data aggregation service for services such as the Microsoft Operations Management Suite.
Fluentd is a data collection and routing service, which means it doesn’t include log shipping or management services. Instead, it integrates with other solutions through plugins, which add support for different inputs and outputs. For example, support for ingesting logs from Amazon CloudFront can be added using the cloudfront-log plugin, while logs can be routed to Elasticsearch using the elasticsearch plugin. This does mean having to build your log management solution essentially from scratch, with Fluentd providing only ingestion and routing services. Fluentd is now a popular replacement for Logstash, turning ELK into EFK.
Fluentd itself is free, but much like the Elastic Stack, it can become expensive over time. Fluentd has a small memory and CPU footprint, but it relies on several other components to create a complete log management solution. This, combined with a lengthy setup and customization process, makes it significantly harder to set up and maintain than other solutions.
4. Sumo Logic
Sumo Logic is a software-as-a-service (SaaS) log management platform that received attention for marketing itself as a cloud-based competitor to Splunk. As a hosted service, Sumo Logic automatically scales to your log volume, claiming to support multiple terabytes of ingested data per day. Sumo Logic also collects metrics from host machines and cloud platforms, letting you track the health of your systems alongside your log data.
Sumo Logic uses agents (called Installed Collectors) to collect and transfer data from host systems. Like Splunk, new functionality can be added to Splunk through add-ons (called apps). Although Sumo Logic’s marketplace isn’t as extensive as Splunk’s, the apps that are available cover a number of popular services and platforms including AWS, Azure, Google Cloud, Docker, and Kubernetes.
Sumo Logic is a strictly cloud-based service, meaning there is no option for on-premise installation. Monthly plans start at $108 per GB per month, with a minimum 3GB of ingestion. This includes 30GB of log data retention. Sumo Logic also offers a free 30 day trial with up to 500 MB of ingestion and 4 GB of retention.
Loggly is a cloud-based log management solution that offers an agentless ingestion service, allowing you to transmit logs directly over HTTP/HTTPS or syslog. Loggly automatically parses a wide range of formats and sources including Docker, AWS, Syslog, Heroku, Windows, and Linux logs. Loggly also offers the ability to create custom parsing rules for unsupported formats.
Loggly’s most defining feature is its field explorer, which lets you search, filter, and summarize logs on a single screen. You can quickly view the frequency of events, select fields and values to filter on, and apply custom search parameters without having to type in a query. These searches can then be converted into alerts for real-time updates and notifications.
Because Loggly is agentless, each log-generating component in your infrastructure must be configured to forward logs to Loggly. Logging Kubernetes and other distributed platforms often means having to use third-party solutions and complex workarounds. This makes Loggly better suited for smaller deployments, or for shipping logs directly from applications. Loggly also does not offer an on-premise solution.
Enterprise plans start at $349/month. Standard plans start much lower at $79 per month, but only offer up to 30 GB/month of ingestion, 30 days of retention, and fewer features.
How to Choose the Best Logging Tools
While most log management solutions offer the same base functionality, each of the tools in this list has their own unique advantages and specialties. The “best” solution depends on what your requirements are, was well as what insights you wish to gain from analyzing your logs. Read this post on How to Choose the Best Log Management System for more guidance.
Before you settle on a solution, give each one a free trial run with your engineers to see the impact it has on your operations.