Defining Log Management and Log Monitoring Objectives

System logs are good for more purposes that many people realize.  In this post, I’ll describe the four broad categories of log usage.

Forensic Record Keeping

Many organizations choose to archive log data for a period of time for future reference.  Generally, the usefulness of keeping archived logs comes from the situation where a system problem is discovered some time in the future, and archived logs can help to piece together the cause of what happened, and can often be used as evidence in court should such a thing become necessary.

Proactive Identification of System Health Problems

Alerting tools can be used to monitor logs streamed from devices and notify operators of a problem, such as a failing disk, low memory, crashing processes, etc.  A big advantage of this is that many serious problems can be remediated before they impact the availability or performance of the system, by removing stale files to add drive space, contacting a vendor for a preemptively hardware replacement and so on.

Proactive Activity Monitoring of Users and Administrators

This is one area of log management that many organizations unintentionally overlook, because the perceived risk is not very high, the administrators are unaware of the options, and more commonly, the view that monitoring the activity of administrators is futile because such monitoring could be bypassed by a skilled administrator.  In separate post, I describe a process to monitor the activities of administrators.

Intrusion Detection

This is arguably the most common use of system logs – looking for evidence of system compromise.  Like system health monitoring, there are many tools on the market that can take in log streams and notify operators or security personnel when an event is detected.

A Note on Protecting Logs

In all cases, it is important to send logs to another system for storage, either real time or by some frequent batch process.  Protecting logs from a failing hard drive, crafty hacker or disgruntled administrator can be achieved by using a hardened syslog server.