What is lurking in your log files?

I found an interesting article on the importance of proactively monitoring your logs here: http://www.esecurityplanet.com/network-security/what-is-lurking-in-log-files.html

3 comments - What do you think?  Posted by mutex - March 15, 2013 at 5:03 pm

Categories: Uncategorized   Tags:

Case Study In Not Managing Logs: DigiNotar

I started a new security podcast recently and in the first episode, I covered the security breech at DigiNotar, the Dutch certificate authority.  One of the prominent findings in the forensic report that was released publicly was the unreliability and unavailability of logs – because, in one case, an administrator erased logs on a server to free up space, and in others, the attacker deleted logs to cover his tracks.  Forwarding ALL logs, whether from Windows systems, firewalls, switches, etc, to a central logging server with very limited access is very key to both forensic analysis after an attack, but also proactive alerting that might indicate an attack is underway, giving time to react before data is lost or damaged.

Be the first to comment - What do you think?  Posted by admin - December 8, 2012 at 4:14 pm

Categories: Logging, Security   Tags:

The Importance Of Remote Logging

Many people will recall the downfall of the Dutch Certificate Authority DigiNotar last year after it was discovered that attackers had compromised the CA’s servers and generated illicit certificates for some major internet sites.  Fox-IT was contracted to investigate the breach and has issued it’s final report.  A notable element in the report was that DigiNotar stored it’s log files on the servers themselves.

The news article on Threatpost includes an except from the report:

“The investigation by Fox-IT showed that all eight servers that managed Certificate Authorities had been compromised by the intruder. The log files were generally stored on the same servers that had been compromised and evidence was found that they had been tampered with. Consequently, while these log files could be used to make inconclusive observations regarding unauthorized actions that took place, the absence of suspicious entries could not be used to conclude that no unauthorized actions took place,”

This really highlights the importance of moving logs to a different server for storage and analysis.  If you aren’t pushing your logs to a logging server, please consider doing so.

Be the first to comment - What do you think?  Posted by mutex - November 1, 2012 at 3:58 pm

Categories: Logging   Tags:

Instructions For Tunnelling Syslog Over SSH Using Syslog-NG

Here’s an interesting article on establishing an ssh tunnel between a client and syslog server for the purpose of securely transmitting syslog messages.

Be the first to comment - What do you think?  Posted by mutex - July 4, 2012 at 5:23 pm

Categories: Logging   Tags: ,

Good Writeup On Implementing A Log Analysis System With Open Source Tools

This post introduces an end to end solution for analyzing logs using:

Rsyslog

Graylog2 (though the post points out that Graylog2 hasspoke shortcomings)

Logstash

graphite

Kibana

Be the first to comment - What do you think?  Posted by mutex - June 17, 2012 at 10:54 pm

Categories: Logging   Tags: , ,

What Happened To Syslog.org?

Regular visitors will no doubt notice that the main part of this site, www.syslog.org, which is the wiki, was down for the past week. I would like to sincerely apologize and explain what happened.

What Happened?

Last week, I opted to upgrade my FreeBSD 8.2 server to CentOS 6. I needed to do this for a few reasons. Due to work pressures, I don’t have a tremendous amount of time to muck with keeping up with the recent spate of FreeBSD vulnerabilities and paches (keeping in mind most of them are with 3rd party components like OpenSSL). The other big reason is that CPanel has sunset support for FreeBSD. I could see signs of the lack of support starting to show.

My datacenter isntalled CentOS on a new drive. I’m not a big Linux guy, but I can be dangerous. After the installation, I had a lot of problems restoring functionality to some sites I host for others. I spent a long, long time debugging those problems. I made a brief check to ensure that my sites, syslog.org among them, were operating normally. Unfortunately, I checked the forum and the blog, but not the actual main site.  I moved on to installing the Config Server Firewall and grappling with a spate of email issues, including a deluge of spam, since my spamassassin customizations hadn’t carried over.

What’s Next?

Well, being one to try to make lemonade out of lemons, I decided to use this “opportunity” to kick start a migration I had been thinking about for a while.  So many years ago, I moved syslog.org to PMWiki with the idea that the site could be somewhat community driven.  What has happened instead is that primarily only spammers are interested in making wiki contributions, PMWiki became a bit of a bear for me to keep updated and the content became stale.

Over the past few years, I have really grown to love WordPress.  I’ve made a number of personal sites and sites for friends/organizations I help out with using WordPress.  So, today formally launches the migration of syslog.org wiki over to WordPress.  There are a lot of details yet to work out, but I am confident it’ll work out.

In the mean time, all of the old content is still available on the wiki site.  I fully intend to take the migration opportunity to update the tools, seek out new tools and other sources of information about logging and log management.

Again, my apologies for any inconvenience while the site was down.

Be the first to comment - What do you think?  Posted by admin - May 13, 2012 at 10:03 pm

Categories: Syslog.org   Tags:

Recent Articles On Logging Standards

There have been a few interesting articles on logging standards recently. First is this one from Dark Reading on the state of logging standards.

Next, is an article from Network World on Mitre’s Common Event Expression (CEE).

Be the first to comment - What do you think?  Posted by admin - April 20, 2012 at 10:45 am

Categories: Logging   Tags:

Samoa’s Time Zone Change Shows The Downside To Using Local Time For Time Stamps in Logs

Naked Security has a good post about Samoa’s move across the international date line.  Using a single time base can be very valuable in recreating the history of an event or incident.  Even if the clocks of all systems are synched using NTP, stitching together logs from systems in different time zones can be tedious, and such a thing generally happens as the result of something bad, where time is of the essence.  The Naked Security post points out that there is even more peril than previously thought in using local time, when the local time changes due to some legal mandate.

Sadly, Samoa is going to completely miss my birthday this year.

Be the first to comment - What do you think?  Posted by admin - December 30, 2011 at 12:45 am

Categories: Log Management, Logging   Tags:

Logging and Syslog Best Practices

In this post, I will cover a basic set of best practices for managing logs. Depending on your specific objectives, regulatory requirements, and business constraints, there are likely to be a number of additional best practices.

  • Forward syslog messages from clients to a secure syslog server.
  • Enable NTP clock synchronization on all clients and on the syslog server. It is very important for all systems
    reporting logs to be using the same time server, so that logs are all synchronized. Without doing this, it can be difficult or impossible to accurately determine the sequence of events across systems or applications.
  • Group “like sources” into the same log file. (i.e. mail server, MTA, spamassassin and A/V scanner all report to one
    file)
  • Use an automated tool to establish a baseline of your logs and escalate exceptions as appropriate.
  • Review your records retention policy, if applicable, and determine if anything kept in logs falls under that policy. If so, establish retention periods based on the records policy.  Legal requirements for keeping logs vary by jurisdiction and application.
  • The “sweet spot” for log retention appears to be one year.  Shorter than 1 year, and it is likely that key data would be unavailable in the wake of a long running attack, and longer than one year is most likely wasting disk space.
  • Include logs and log archives in a standard backup process for disaster recovery.
  • Change read/write permissions on logs files so they are not accessible to unprivileged user accounts.

Have more suggestions for logging best practices? Post them in a comment below.

16 comments - What do you think?  Posted by admin - October 4, 2010 at 9:16 pm

Categories: Compliance, Log Management, Logging, Policy   Tags:

Reading Logs From A File In Syslog-NG

I had previously written a little snippet on how to pull logs in from a file, however there is a substantial amount more to consider when configuring syslog-ng to read from a file, so I have dedicated this post to reading logs from a text file.

The basic structure for reading logs from a text file looks like this:

source s_file {
file(“/var/log/app.log”);
};

destination d_messages{
file(“/var/log/messages”);
};

log {
source(s_file);
destination(d_messages);
};

The source file driver has a lot of options which can you help to tailor the import of logs into syslog.

One of the first, and most important settings to consider when reading logs in from the file is the “flags(no-parse)” option on the source file statement.  By default, syslog-ng expects the lines in the text file being read to be in the format of a syslog message, and will attempt to parse out the relevant fields, like date/time, severity, program, etc.  However, most applications that write custom log files don’t use a standard syslog format, and so we need a way to tell syslog-ng that the whole log record in our log file is the “log message”, and not a syslog-formatted message.  We accomplish this with the “flags(no-parse)” option, which looks like this in the config file: Read more…

15 comments - What do you think?  Posted by admin - July 25, 2010 at 11:28 am

Categories: Logging   Tags:

Recent Posts in the Syslog Forum

RSS Error: A feed could not be found at http://www.syslog.org/forum/.xml/?type=rss. A feed with an invalid mime type may fall victim to this error, or SimplePie was unable to auto-discover it.. Use force_feed() if you are certain this URL is a real feed.

Next Page »