SenseDeep 2.1 — Streamlined Navigation

Searching log files is often the proverbial searching for a “needle in the haystack”. However, SenseDeep’s streamlined log viewer that makes it fast and easy to navigate your logs and find those application errors or important log events.

The SenseDeep viewer is the result of many hours of painstaking development to create a log viewer that is unrivaled in speed and is a joy to use.

If you prefer a video instead, watch SenseDeep Log Viewer.

Log Capture and Storage

SenseDeep captures log data via a dedicated lambda that runs in your account called the SenseDeepWatcher. This lambda subscribes to logs of interest and stores log data in a high-performance DynamoDB log database in your account.

The SenseDeep app running in your browser directly reads log data from the log database for immediate presentation. The SenseDeep viewer is extremely fast in locating, reading and presenting log data of interest.

SenseDeep provides open access to this log database and publishes the log data schema so you create your own log analysis capabilities. This is an open, transparent architecture for your logging needs.

Viewer Features

The SenseDeep log viewer supports:

• Automatic aggregation of log streams into a unified view.
• Combine and correlate multiple logs into a unified single view.
• Automatic, real-time live tail display of new log events — you never need to click reload again.
• Streamlined log navigation bar with date scrubber, filter, and key operations at your finger tips.
• Filter by request IDs to isolate individual requests.
• Smart log parsing to map log fields to display columns.
• Drag and drop to reorder and resize display columns.
• Quick search forwards and backwards in filtered results.
• Current row highlighting and navigation.
• Easy timezone switching between local and UTC.
• Color coded expanded JSON events.

Date Scrubber

The timeline scrubber permits you to jump to the start or end of the log file or any arbitrary date in between. Changing the scrubber modifies the highlighted current line in the log file. Clicking on the scrubber date box will display a date and time edit dialog.

Viewer Navigation Bar

The SenseDeep Viewer has a new navigation bar with the most common commands at your finger tips. These are also mapped to keyboard shortcuts.

Expand Records

The Expand / Collapse icon will expand all log entries and display all fields vertically. Clicking the icon will collapse any expanded fields. You can also type the ENTER key or Command-Click / Control-Click on a row to expand or collapse individual records.

If a log entry contains JSON, it will be expanded and color coded. Properties will be displayed in sorted order. SenseDeep uses an abbreviated JSON presentation where quotes are removed from strings and end-of-line commas are also removed for clarity. If you export the current record to the clipboard (Cmd-E), the full JSON record with quotes and commas will be exported.

Cut Down Log Events

Sometimes you may wish to only focus on new log data and not be distracted by any historical or prior log data. For example, you may be developing a Serverless Lambda application and you want to trigger a specific log condition and be able to easily locate that new log data. The SenseDeep Cut Log Data can help.

The cut log icon will discard all prior log data and display only new log data. This is useful when used in conjunction with the Live Tail option below. Clicking the cut icon a second time will restore the cut log entries.

Live Tail

The Live Tail icon puts the viewer into “Live Tail” mode where new log data is automatically fetched and appended to the end of the log view. If you have scrolled to the end of the view, the Viewer will automatically scroll as new log data is fetched.

Select Fields

The select fields icon will display the select fields icon. This new release has two important enhancements. SenseDeep will automatically select a default set of top level fields when the view is first created. Thereafter, you can add or remove fields. If you select the “Freeze” checkbox, the view fields will be fixed (frozen) and not be modified as new fields are discovered in the data stream.

By default, the Viewer orders fields in sorted order. If you add a field via the select fields dialog, it will be added to the end of the table. You can drag and drop Viewer field columns to reorder the fields.

Search

The new Search dialog permits you to search within selected log records for specific text. You can search forwards, backwards or jump to the current line.

Not to be confused with view filters, which select which log records to display, the search dialog selects the current record to jump to.

The search dialog can also be invoked via the Cmd-F keyboard shortcut.

Context and Navigation

Log files can be confusing places. Gigabytes of log data that is often confusingly similar. Find an application error and then maintaining context as you look around at related log data can be difficult. SenseDeep helps this situation by maintaining a “current” line context and by highlighting that line in blue to make it easily visible.

You can navigate from the current line by using Arrow keys up and down, or by clicking on any line to make that the current line. Use Cmd-J to jump back to the current line. The Search dialog (Cmd-F) can be used to search for a specific text pattern and jump the current line to the first such occurrence of the pattern.

SenseDeep preserves your context as you expand log entries, select new fields or resize or reorder columns.

Modifying and Saving Views

You can make changes to the log view presentation via the icons or via the Modify View icon. When clicked, the Modify View icon will display a slide-out panel with detailed options to filter the log events displayed.

To filter log events, enter a text pattern, regular expression or text field expression to select the desired log events to display. You can specify text patterns that must match and also patterns to exclude log events from display.

You can also define the time range from which to select events by using an absolute start / end time or by a relative time period.

The expand, live tail and hide service events options are also selectable from the modify view panel.

You can save the view to a new name by selecting New View from the name pulldown and then entering a new name. You can select the view to display by clicking on the log title in the top left of the viewer.

You can create a dashboard widget that will invoke the viewer and display the desired view.

When you modify any viewer presentation option, a Save button will appear. Clicking save will persist your changes as the new default presentation for that view. If you are a member of a shared organization your changes will be shared with all team members.

Clipboard

To copy the current record to the clipboard for export to another application, use the Cmd-E key to export the record to the clipboard.

Short Cuts

In summary, here are the keyboard shortcuts supported by the SenseDeep viewer:

• Cmd-E — Export the current record to the clipboard
• Cmd-F — Find text in the log view
• Cmd-J — Jump to the current record
• Cmd-X — Cut
• Cmd-Click &mdash ; Expand record
• Up / Down arrow keys — Move the current row up or down
• Enter — expand the current row

On Windows, use Control instead of Cmd.

Summary

The SenseDeep viewer is a powerful tool to quickly and efficiently navigate your log data. By transparently aggregating log streams and groups into a unified view, you can more easily find those troublesome log events.