Secure Log Data so You Never Need to Redact

Most 3rd party logging solutions ship your log data offsite to their proprietary log storage mechanisms. However, log data often contains sensitive information and identifiers. Protecting your log data is critical for the security of your account and you are relying on these vendors to never expose or have accidents with your log data.

SenseDeep is different. Your log data never leaves your account and is stored in a DynamoDB database in your account over which you have full control. When you use the SenseDeep app, your browser directly accesses the log data in your account.

The result is secure log storage and analysis with fast, predictable access to critical event and log information.

SenseDeep Watcher and Log Capture

SenseDeep captures log data by automatically subscribing to CloudWatch log groups. To do this, SenseDeep runs a small Lambda function called the SenseDeepWatcher in your account. The Watcher receives log data and stores it in a SenseDeep DynamoDB database together with your Alarm and Alert configuration.

The Watcher and the DynamoDB table run inside your AWS account in your designated regions. This offers the highest level of security as your log data never leaves your account. Log data often has sensitive resource identifiers, addresses and other secrets that you do not wish to have exposed. By keeping this information in your account and never sending it over the wire — your service security is strengthened.

Log access performance is also enhanced and latencies are reduced because log data does not have far to travel from where you generate it, to its final resting place.

Open Access

While SenseDeep provides log capture, a fast log viewer and alarms based on your log data, there are many possible needs that cannot be foreseen. To meet these needs, SenseDeep publishes the schema for the log database so that you can extend SenseDeep and create your own custom log analysis capabilities. This is an open, transparent architecture for your logging needs.

The schema for the DynamoDB SenseDeep table is defined at:

• https://github.com/sensedeep/sensedeep/blob/master/schema/index.js

This table schema is defined using OneTable which is a DynamoDB access library for single-table designs. You can use OneTable in your code with this schema to freely access and utilize your log data in your account. Read Open Architecture for more details.

Scalable

The SenseDeep logging architecture is 100% serverless. As your log volume increases, AWS Lambda will scale the Watcher as required to capture all your log data. As your logging load decreases, the Lambda service automatically adjusts and scales down the Watcher concurrency.

This enables SenseDeep to offer pricing plans that have no log ingestion limits. You are not capped on the volume of logs or on the amount of log data captured. As your log load increases, the Watcher Lambda and SenseDeep DynamoDB table will scale predictably with a very low cost.

Summary

SenseDeep has the most secure logging solution by protecting your log data and ensuring it never leaves your account.