Archive for February, 2010

Designing A Log and Event Monitoring Program

Ultimately, as with all IT security programs, log monitoring programs are designed to address risks to data confidentiality, integrity and availability.  Risks come in many types:

• Hardware failure
• System compromise
• User error
• Rogue administrator

An organization’s program around log & event monitoring needs to be based on the specific risks that exist in that organization.  Consider two these two scenarios:

Scenario 1: The IT department of a large public corporation manages a large number of critical business systems, including systems that contain the company’s financial information, payroll information and client data.  This company cares deeply about the confidentiality, integrity and availability of it’s applications.

Scenario 2: A small Internet-based retailer of custom soaps is owned by a family.  Dave, a member of the family that owns the business, is technically savvy and has colocated a server to run the retailer’s web site.  Dave manages the server and the web site.  Dave also cares deeply about the confidentiality, integrity and availability of his company’s web site. Read more…

Running Syslog-NG on Windows

This post describes running syslog-ng as a server on Windows.  In another post, we describe how to send Windows Event Logs to syslog.

There are many great commercial syslog servers for Windows.  There are not many options for those looking for a free alternative.  One option is Aonaware.  Another option is to install syslog-ng through cygwin.  Cygwin is a Linux-like environment run inside a windows command shell.  Cygwin runs on all current desktop and server versions of Windows.  In this post, we will walk through setting up syslog-ng on a windows host.

1. First, visit the Cygwin website to download the setup.exe application.  Save, then run setup.exe.
2. Choose “Install From Internet”
3. Select the directory to install into and the user to install for (leave this as “all users”).
4. Enter the directory for local packages. Accepting the default location is fine.
5. Choose your Internet connection type (direct or proxy)
6. Select a site to download from.  Any one should be fine.
7. At the “install packages” window, type is “syslog” in the search box.  You will see “Admin” below.  Expand the admin section, and you will see syslog-ng.  Click the word “skip” until you see 3.0.1 (or whatever the latest supported version is).
8. Also choose the following packages:
   1. Admin/cygrunsrv
   2. Editors/VIM
   3. Gnome/glib
9. Finish the installation. Read more…

Pot Of Syslog-NG Tricks Version 1

Fixing Duplicate Date and/or Hostname Problems

Some devices send syslog messages with improperly formatted headers, which can cause syslog-ng to append a new set of header information, meaning that the host name and/or date appear twice in the logs.  A simple way to solve this is using a template:

source s_net { udp();};
destination d_file { file(“/var/log/file.log” template(“$MSG\n”)); };
log {source(s_net); destination(d_file); };

The template function in the destination definition tells syslog-ng to discard the header from the received message and only keep the actual message data, which is contained in the variable $MSG.   Syslog-ng inserts the proper header, using the date the log was received and the host the log came from, resulting in a normal syslog message. Read more…

Defining Log Management and Log Monitoring Objectives

System logs are good for more purposes that many people realize.  In this post, I’ll describe the four broad categories of log usage.

Forensic Record Keeping

Many organizations choose to archive log data for a period of time for future reference.  Generally, the usefulness of keeping archived logs comes from the situation where a system problem is discovered some time in the future, and archived logs can help to piece together the cause of what happened, and can often be used as evidence in court should such a thing become necessary.

Proactive Identification of System Health Problems

Alerting tools can be used to monitor logs streamed from devices and notify operators of a problem, such as a failing disk, low memory, crashing processes, etc.  A big advantage of this is that many serious problems can be remediated before they impact the availability or performance of the system, by removing stale files to add drive space, contacting a vendor for a preemptively hardware replacement and so on.

Proactive Activity Monitoring of Users and Administrators

This is one area of log management that many organizations unintentionally overlook, because the perceived risk is not very high, the administrators are unaware of the options, and more commonly, the view that monitoring the activity of administrators is futile because such monitoring could be bypassed by a skilled administrator.  In separate post, I describe a process to monitor the activities of administrators.

Intrusion Detection

This is arguably the most common use of system logs – looking for evidence of system compromise.  Like system health monitoring, there are many tools on the market that can take in log streams and notify operators or security personnel when an event is detected.

A Note on Protecting Logs

In all cases, it is important to send logs to another system for storage, either real time or by some frequent batch process.  Protecting logs from a failing hard drive, crafty hacker or disgruntled administrator can be achieved by using a hardened syslog server.

Configuring SUDO for Effective Activity Monitoring Via Syslog

I have discussed in previous posts the importance of administrators using SUDO to provide individual accountability.  SUDO provides command-by-command accounting of actions performed by administrators, with logs sent as standard syslog events looking like this:

Feb  4 19:23:23 bsd sudo:    jerry : TTY=pts/0 ; PWD=/usr/home/jerry ; USER=root ; COMMAND=/bin/ps -x
Feb  4 19:23:34 bsd sudo:    jerry : TTY=pts/0 ; PWD=/usr/home/jerry ; USER=root ; COMMAND=/usr/bin/vi /etc/passwd
Feb  4 19:23:59 bsd sudo:    jerry : TTY=pts/0 ; PWD=/usr/home/jerry ; USER=root ; COMMAND=/usr/bin/tail -100 /var/log/messages

We can see pretty clearly all the actions I took above: the user “jerry” performed a number of actions, including one that is potentially concerning: vi /etc/passwd.  The action on /etc/passwd requires some investigation.

First, we need to be sure that an administrator can’t cover his tracks by deleting logs.  This is best accomplished by streaming the logs to a hardened syslog server, where the administrator doesn’t have the ability to delete logs.  Read more…