Enterprise — Log Forwarding

What happens if you must use an Enterprise logging solution such as Splunk, due to a corporate directive, but you also want to use a faster, simpler, more effective serverless troubleshooter for your own needs?

In the past, you were out of luck.

You could use either the enterprise logger and keep corporate happy, or you could use your serverless troubleshooter and defy the corporate overlords.

Now, there is a better way.

The Problem

AWS CloudWatch only permits one log subscription per log group to automatically ingest log data. So if you must use that subscription to capture and forward that logging data to an enterprise logging solution, it is difficult to capture that data for more dedicated Serverless monitoring solution such as SenseDeep.

Some products use polling as a workaround to capture log data, but that is slow and costly and not an effective solution.

The Solution

SenseDeep solves this problem by running a small Lambda, called the Watcher, in your account. The SenseDeep Watcher captures Lambda and CloudWatch log data and monitors your service to trigger alarms for potential issues.

The Watcher can also be configured to forward log data to an enterprise logging solution for archiving and permanent capture. The Watcher can thus replace the traditional enterprise logging capture mechanism.

The Watcher is a highly optimized log capture agent and is extremely effective as you are only capturing the log data once, yet you are able to utilize the data for two services: enterprise logger and SenseDeep Serverless monitoring.

The Watcher can dynamically subscribe and unsubscribe matching CloudWatch logs via tag or pattern matching. This means you can configure the forwarding of logs once and it will automatically track new Lambdas as your service evolves.

Configuring Log Forwarding

To configure log forwarding in SenseDeep, you create a special “Relay” alarm for the log data you wish to forward.

Select Alarms from the sidebar menu and then Add Alarm. Select the Rely alarm type and enter the required details.

You can select the log groups to forward via an explicit “list” or by matching patterns that use regular expressions or AWS tags.

The regular expression and tag matching utilize two patterns. One to select a set of resources to include and a second for resources to exclude. This gives maximum flexibility in defining the set of log groups to forward.

Using regular expressions or tag matching enables dynamic matching such that new log groups will be automatically subscribed as they are created or as their AWS tags are modified. The Watcher listens for AWS CloudWatch log group creation and AWS tag modification events and responds by subscribing or unsubscribing as required by your relay requirements.

Your enterprise logger will typically require an API key or authorization token be included in the log forward request. You can specify this on the URL or as a custom header in the headers section. Consult your enterprise logger for details.

Summary

SenseDeep log forwarding is a highly efficient way to capture and forward log data to your enterprise logging solution. It dynamically subscribes to new log groups and will capture log data, check for triggered alarms and then forward that data to your enterprise logger.

Yes, you can now have your cake and eat it too. You can use both an enterprise logger and a dedicated serverless solution without penalty.