Source code management (SCM) is a core component of DevOps. In addition to storing and sharing source code, SCM tools maintain an ongoing history of changes. Reviewing this history provides numerous insights into your development process, including:
- How often code changes are submitted
- The impact of changes on application performance
- Which changes result in errors, bugs, or broken builds
As development becomes an increasingly automated process, monitoring changes to applications will become increasingly important. In addition, having the ability to trace problems back to their source will be essential to engineering teams. For example, if a particular change causes an entire deployment to fail, we need to know exactly which change it was and when it was integrated into our source code. This requires us to not only collect logs from the application, but also from the SCM tool.
With LogDNA, you can monitor logs from GitHub the same way you monitor logs from your applications and infrastructure. LogDNA’s GitHub integration lets you view events from your GitHub repositories—such as commits, branches, pull requests, and releases—in the LogDNA web app. You can then use LogDNA’s full suite of tools to search, filter, and visualize your log data for in-depth analysis.
In this post, we’ll explain how you can use GitHub integration to effectively monitor and analyze activities in your GitHub repository.
What GitHub Events can show up in LogDNA
LogDNA categorizes GitHub events into four different types, which you can enable or disable independently for each repository (except for core events):
- Core events include basic repository actions such as pushing commits, submitting pull requests, creating releases, and forking.
- Extended events include GitHub-specific actions such as starring a repository, organization and team actions, and creating a GitHub Pages site.
- Issues events are generated whenever a GitHub Issue is created or modified.
- Deployment events are generated by GitHub Deployments.
If an enabled action is performed in your repository, GitHub sends a Webhook containing the event’s details to LogDNA. The event appears in your log feed with “GitHub” as the source and your repository name as the App.
For example, we created a repository named “github-integration-test” and pushed a simple Python application to it. This created the following event in LogDNA containing the repository name (“github-integration-test”), affected branch (“master”), user (“logdna”), message (“Initial commit”), and list of modified files:
Next, let’s look at a few scenarios where this type of data would be useful.
Monitoring and Reviewing Code Activity
LogDNA records a continuous chronological stream of events for each repository where GitHub integration is enabled, helping you stay on top of changes to your applications. While this is also available in GitHub, LogDNA makes it easy to consolidate, scroll, and search through your code history.
For example, say you want to review changes made on a certain date. You could open your repository in GitHub and click through the commit history until you come across the correct date. You could also use git log and scroll to the correct date. Alternatively, you can open LogDNA, enter “app:<your repository name>” in the search box, and enter the exact date that you wish to search in the Timeframe box.
This lets you quickly and easily review changes, search through older commits, and monitor ongoing changes to source code.
Consolidating Application and Event Logs
GitHub integration allows you to bridge events from your repository with logs from your application. By sending logs from both GitHub and your application to LogDNA, you can gain a complete view of your application from a single interface. This is particularly useful for continuous integration and continuous delivery (CI/CD) pipelines, where code changes are often rapidly and automatically deployed into production.
For example, let’s create a new view specifically for logs relating to our application. An easy way to correlate application logs and GitHub logs is to tag our application logs with the same “app” name as our GitHub repository. This way, we can select the app name from the “All Apps” drop-down to show both types of logs.
Here, we created a GitHub Issue to change the default hostname used by the LogDNA client to “logdna-node-1”. After pushing the changes and running the new app, we can immediately see the impact on our logs:
What Broke the Build?
Facebook popularized the phrase “move fast and break things,” which means bugs will be there and things will break. If a change results in a production error, we need to detect the error and the change that introduced it as quickly as possible.
For example, what if our latest change caused a new error message to appear in the logs? Previously, we’d need to manually match dates between our application logs and commit history to figure out which commit caused the problem. By combining application logs and GitHub events into a single view, we can simply view our error log in context to find out which event caused the change. This makes it easy to trace specific problems back to the latest or commit for faster troubleshooting.
There’s much more to LogDNA’s GitHub integration than the use cases we covered in this blog. In addition to tracking commits and issues, GitHub integration lets you log deployment events, releases, forks, pull requests, organization and team events, and even events surrounding GitHub Pages sites.