By:

Read Time: 8 min

LogDNA is excited to be a Silver Member of Cloud Foundry this year. If you’re at the North America Summit 2019 this week, drop by our booth B7 we would love to meet you.

Cloud Foundry Application Runtime is an open source platform as a service (PaaS) for running applications and services. Frequently called simply “Cloud Foundry,” the Cloud Foundry Application Runtime (CFAR) is one of many interoperable projects within the Cloud Foundry family. For the purposes of this post, “Cloud Foundry” refers to the Application Runtime.

It allows organizations to automatically deploy, manage, and scale networked workloads on any infrastructure, including cloud IaaS providers and on-premise. Its design is similar to that of the serverless architecture, in that developers can deploy code or binaries directly to the platform. Cloud Foundry handles packaging, deploying, networking, scaling, and load balancing these workloads across the infrastructure, leaving developers to focus on building the workloads.

In this post, we’ll show you how to use our Cloud Foundry integration to effortlessly ship logs from your Cloud Foundry deployments to LogDNA. We assume that you already have a working Cloud Foundry deployment with actively running apps, as well as a LogDNA account. If you’re new to Cloud Foundry, you can learn more at the Cloud Foundry documentation site.

How Logging Works in Cloud Foundry

The Cloud Foundry logging system is called Loggregator. Loggregator collects logs and metrics from apps, aggregates them, and routes them to any clients or services that request them. Loggregator automatically collects logs from any and all apps, as long as those apps print their logs to standard output.

Loggregator has the following architecture:

  1. Apps generate logs which are collected by Sources.
  2. Sources forward logs to Metrons, which are agents running on the same node as the Sources they log.
  3. Metrons forward logs to Dopplers, which are servers that collect and store logs in temporary buffers.
  4. A Traffic Controller handles requests from external logging clients and services. The Traffic Controller exposes a Firehose, which is a WebSocket endpoint capable of streaming logs and metrics to remote services.

The Firehose is the most important aspect of this architecture, since it creates an access point for applications and services (called nozzles) to access the log stream. You can learn more about nozzles from the Cloud Foundry nozzle tutorial. In this next section, we’ll explain how LogDNA uses the Firehose to ingest logs.

Integrating LogDNA with Cloud Foundry

LogDNA provides two methods of integrating with Cloud Foundry:

  1. Creating a user-provided service instance and binding it to an individual app.
  2. Using the LogDNA nozzle to log your entire Cloud Foundry deployment.

The next steps assume that you already have the Cloud Foundry CLI installed and registered with your Cloud Foundry organization.

1. Creating a User-Provided Service Instance

A user-provided service instance allows you to stream logs from a Cloud Foundry app to LogDNA via syslog. This is the easier method to set up, but it will only log apps that you bind the service instance to.
Start by logging into your LogDNA account and navigating to Integrations. Select the Cloud Foundry integration and select Provision a Syslog Port. You will receive a new LogDNA syslog endpoint, which you can use to connect your service:

The first command creates a new service named my-logs that points to the LogDNA endpoint. The second command binds my-logs to your Cloud Foundry app (make sure you replace <Cloud Foundry app name> with the name of your app). The final command restarts your app with the new service bound. Now, any logs that your app writes to standard output will appear in LogDNA.

2. Using the LogDNA Nozzle (Recommended)

The LogDNA nozzle streams events from your entire Cloud Foundry deployment to LogDNA. The nozzle runs as an app within Cloud Foundry, making it easy to deploy and manage. In minutes, all of your Cloud Foundry app logs will begin appearing in LogDNA.

Start by downloading the latest release of the LogDNA nozzle:

$ git clone https://github.com/logdna/logdna-cloudfoundry.git
$ cd logdna-cloudfoundry

Before connecting any nozzle to the Firehose, you must first authenticate the nozzle as a User Account and Authentication (UAA) client. Access to the Firehose is controlled by the doppler.firehose scope. To learn more about nozzle authentication and permissions, read the Cloud Foundry nozzle documentation.

Once your nozzle is authorized, you can push it to Cloud Foundry. We’ll name ours firehose-to-logdna so that it’s easy to identify in the app list. The –no-start parameter specifies that we don’t want to run the app after pushing it:

$ cf push firehose-to-logdna --no-start

Next, we need to set several environment variables. These provide the app with the data needed to authenticate not just to with Firehose, but with LogDNA. They include:

VariableDescription
API_ENDPOINTYour Cloud Foundry system domain.
DOPPLER_ENDPOINTYour Cloud Foundry system domain.
INGESTION_KEYYour LogDNA ingestion key (available here).
FIREHOSE_CLIENT_IDThe ID of the LogDNA UAA client.
FIREHOSE_CLIENT_SECRETThe secret associated with the UAA client.
SYSLOG_ENDPOINTAn optional endpoint for a syslog server.

You can set each of these variables using the Cloud Foundry CLI:

$ cf set-env firehose-to-logdna API_ENDPOINT [your API_ENDPOINT]
$ cf set-env firehose-to-logdna DOPPLER_ENDPOINT [your DOPPLER_ENDPOINT]
$ cf set-env firehose-to-logdna INGESTION_KEY [your INGESTION_KEY]
$ cf set-env firehose-to-logdna FIREHOSE_CLIENT_ID [your FIREHOSE_CLIENT_ID]
$ cf set-env firehose-to-logdna FIREHOSE_CLIENT_SECRET [FIREHOSE_CLIENT_SECRET]
$ cf set-env firehose-to-logdna SYSLOG_ENDPOINT [your optional SYSLOG_ENDPOINT]

Finally, push the nozzle to your Cloud Foundry deployment:

$ cf push firehose-to-logdna

In a few moments, logs from your Cloud Foundry applications will begin appearing in LogDNA.

Conclusion

With LogDNA, you can quickly and easily log your entire Cloud Foundry deployment in a matter of minutes. To get started, log into your LogDNA account and navigate to the Cloud Foundry integrations page. If you wish to learn more about our Cloud Foundry Firehose nozzle, visit the project’s GitHub page. And if you have any questions, please contact us.



About Thu Nguyen

Thu Nguyen is a technical writer who cares deeply about human relationships.

RELATED POSTS

LogDNA Events in June

LogDNA at Velocity A few notable events in June that we attended were 1. Velocity Conference It was a treat to attend the Velocity and...

Events
Cf Days Webbanner 1600x500 V1 Ac Nologo 01 1200x375

Open Ecosystems, Interoperability + Multi-Cloud: Risk & Reward for Developers Webinar

Our VP Product, Peter Cho hosted a webinar with CloudFoundry last week to share his pragmatic approach in this golden age of developer tools, on-premise...

Events
Cloud Foundry

RSVP for our Upcoming DevOps Webinar

In our next webinar, Peter Cho, VP Product at LogDNA will be presenting: Open Ecosystems, Interoperability + Multi-Cloud: The Risks, Rewards & Trade-offs for Developers...

Events