individual files

Newbie
Posts: 10

individual files

After playing around with syslog-ng all afternoon it's config process leaves something to be desired. Needless to say I’m a little confused Huh

and frustrated. I have managed to get it running on most of my Linux boxes and it to accept data from a couple of them still running syslog which is what I want. The issue I’m facing is the config file I found do to the wonders of Google is splitting the information based on host but not into separate files like I’m use to with syslog aka cron, messages, syslog etc.. I'll post the file I’m using below any help would be useful

Bellow is the file i'm using on my log host.

Code:

#
# Syslog-ng configuration file. Originally written by anonymous (I can't find
# his name) Revised, and rewrited by me (SZALAY Attila <sasa@debian.org>)
# revised again by Nate Campi <nate at campin dot net>

###############################################################
# First, set some global options.

options {
use_fqdn(yes);
use_dns(yes);
dns_cache(yes);
keep_hostname(yes);
long_hostnames(off);
sync(1);
log_fifo_size(1024);
};

###############################################################
#
# Logs may come from unix stream, and UDP:514
#
source src {
pipe("/proc/kmsg");
unix-stream("/dev/log");
internal();
udp(ip("192.168.2.3") port(514));
# tcp(port(5140) keep-alive(yes));
# tcp(ip("192.168.2.3") port(5140) keep-alive(yes));
};

###############################################################
# After that set destinations.

# First some standard logfile
#
destination authlog { file("/var/log/auth.log"); };
destination syslog { file("/var/log/syslog"); };
destination cron { file("/var/log/cron.log"); };
destination daemon { file("/var/log/daemon.log"); };
destination user { file("/var/log/user.log"); };
#destination debug { file("/var/log/debug"); };
destination messages { file("/var/log/messages"); };

# Virtual console.
#
destination console_all { file("/dev/tty12"); };

##########################################

# Here's the filter options. With this rules, we can set which
# message go where.

filter f_authpriv { facility(auth, authpriv); };
filter f_syslog { not facility(auth, authpriv) and not facility(mail); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
#filter f_lpr { facility(lpr); };
#filter f_mail { facility(mail); };
filter f_user { facility(user); };
filter f_uucp { facility(cron); };

#filter f_news { facility(news); };

#filter f_debug { not facility(auth, authpriv, news, mail); };
filter f_messages { level(info .. warn)
and not facility(auth, authpriv, cron, daemon, mail, news); };
filter f_emergency { level(emerg); };

filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };

filter f_cnews { level(notice, err, crit) and facility(news); };
filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };

###############################################################
#
# log statements actually send logs somewhere, to a file, across the network, etc
#

log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_authpriv); destination(syslog); };
log { source(src); filter(f_syslog); destination(syslog); };
log { source(src); filter(f_cron); destination(cron); };
log { source(src); filter(f_daemon); destination(daemon); };
log { source(src); filter(f_daemon); destination(messages); };
log { source(src); filter(f_kern); destination(messages); };
log { source(src); filter(f_user); destination(messages); };
#log { source(src); filter(f_debug); destination(debug); };
log { source(src); filter(f_messages); destination(messages); };
log { source(src); filter(f_cnews); destination(console_all); };
log { source(src); filter(f_cother); destination(console_all); };

## set up logging to loghost
#destination loghost {
# tcp("10.0.0.1" port(514));
#};

# send everything to loghost, too
#log {
# source(src);
# destination(loghost);
#};

#
# automatic host sorting (usually used on a loghost)
#

# set it up
destination std {
file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY_$HOST_$YEAR_$MONTH_$DAY"
owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)
);
};

# log it
log {
source(src);
destination(std);
};

###############################################################

August 10, 2008, 05:38:09 pm

#1

mutex

mutex

Administrator
Newbie
Posts: 782

Re: individual files

What is the end result that you are looking for?

August 10, 2008, 05:45:51 pm

#2

spider

spider

Newbie
Posts: 10

Re: individual files

hopeing for a folder per host per year per month. for example:

/var/log/peter.domain.com/2008/08/

then your standard logs. secure, messages, cron, spooler, syslog etc.

August 10, 2008, 05:51:20 pm

#3

mutex

mutex

Administrator
Newbie
Posts: 782

Re: individual files

The problem is that your log() statements were going to /var/log/* files in this section:
destination authlog { file("/var/log/auth.log"); };
destination syslog { file("/var/log/syslog"); };
destination cron { file("/var/log/cron.log"); };
destination daemon { file("/var/log/daemon.log"); };
destination user { file("/var/log/user.log"); };
#destination debug { file("/var/log/debug"); };
destination messages { file("/var/log/messages"); };

So, I updated the log statements to use the "std" destination, which should do what you are looking for.

Code:

#
# Syslog-ng configuration file. Originally written by anonymous (I can't find
# his name) Revised, and rewrited by me (SZALAY Attila <sasa@debian.org>)
# revised again by Nate Campi <nate at campin dot net>

###############################################################
# First, set some global options.

options {
use_fqdn(yes);
use_dns(yes);
dns_cache(yes);
keep_hostname(yes);
long_hostnames(off);
sync(1);
log_fifo_size(1024);
};

###############################################################
#
# Logs may come from unix stream, and UDP:514
#
source src {
pipe("/proc/kmsg");
unix-stream("/dev/log");
internal();
udp(ip("192.168.2.3") port(514));
# tcp(port(5140) keep-alive(yes));
# tcp(ip("192.168.2.3") port(5140) keep-alive(yes));
};

###############################################################
# After that set destinations.

# First some standard logfile
#
destination authlog { file("/var/log/auth.log"); };
destination syslog { file("/var/log/syslog"); };
destination cron { file("/var/log/cron.log"); };
destination daemon { file("/var/log/daemon.log"); };
destination user { file("/var/log/user.log"); };
#destination debug { file("/var/log/debug"); };
destination messages { file("/var/log/messages"); };

# Virtual console.
#
destination console_all { file("/dev/tty12"); };

##########################################

# Here's the filter options. With this rules, we can set which
# message go where.

filter f_authpriv { facility(auth, authpriv); };
filter f_syslog { not facility(auth, authpriv) and not facility(mail); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
#filter f_lpr { facility(lpr); };
#filter f_mail { facility(mail); };
filter f_user { facility(user); };
filter f_uucp { facility(cron); };

#filter f_news { facility(news); };

#filter f_debug { not facility(auth, authpriv, news, mail); };
filter f_messages { level(info .. warn)
and not facility(auth, authpriv, cron, daemon, mail, news); };
filter f_emergency { level(emerg); };

filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };

filter f_cnews { level(notice, err, crit) and facility(news); };
filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };

###############################################################
#
# log statements actually send logs somewhere, to a file, across the network, etc
#

log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_authpriv); destination(std); };
log { source(src); filter(f_syslog); destination(std); };
log { source(src); filter(f_cron); destination(std); };
log { source(src); filter(f_daemon); destination(std); };
log { source(src); filter(f_daemon); destination(std); };
log { source(src); filter(f_kern); destination(std); };
log { source(src); filter(f_user); destination(std); };
#log { source(src); filter(f_debug); destination(std); };
log { source(src); filter(f_messages); destination(std); };
log { source(src); filter(f_cnews); destination(console_all); };
log { source(src); filter(f_cother); destination(console_all); };

## set up logging to loghost
#destination loghost {
# tcp("10.0.0.1" port(514));
#};

# send everything to loghost, too
#log {
# source(src);
# destination(loghost);
#};

#
# automatic host sorting (usually used on a loghost)
#

# set it up
destination std {
file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY_$HOST_$YEAR_$MONTH_$DAY"
owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)
);
};

# log it
log {
source(src);
destination(std);
};

###############################################################

August 10, 2008, 06:22:32 pm

#4

spider

spider

Newbie
Posts: 10

Re: individual files

hmm not sure i see a change in the output produced

Currently it's producing the desire path automaticly
/var/log/HOSTS/peter.domain.com/2008/08/
instead of spliting into multiple log files it's creating one file for the date. maybe what i'm after isnt possible at this rate i could of set a cronjob to just copy the files at a set interval lol

i guess what i'm after would be if i ran an ls /var/log/HOSTS/peter.domain.com/2008/08 i'd see a list of log files

Code:

4.0K -rw------- 1 root root 752 2008-08-10 20:06 auth.log
4.0K -rw------- 1 root root 1.2K 2008-08-10 17:46 btmp
0 -rw-r--r-- 1 root root 0 2008-08-10 14:36 cron
0 -rw-r--r-- 1 root root 0 2008-08-10 17:22 cron.log
0 -rw-r--r-- 1 root root 0 2008-08-10 17:22 daemon.log
4.0K -rw------- 1 root root 2.8K 2008-08-10 18:26 debug
8.0K -rw-r--r-- 1 root root 6.6K 2008-08-10 14:36 dmesg
0 -rw-r----- 1 root root 0 2002-04-12 23:47 faillog
4.0K -rw-r--r-- 1 root root 292 2008-08-10 20:06 lastlog
0 -rw-r----- 1 root root 0 2002-04-06 19:13 maillog
12K -rw------- 1 root root 8.2K 2008-08-10 20:05 messages
0 -rw-r--r-- 1 root root 0 2008-08-10 18:22 ntpd
0 -rw-r----- 1 root root 0 1994-05-09 03:06 secure
0 -rw-r----- 1 root root 0 2002-03-09 00:29 spooler
12K -rw------- 1 root root 8.6K 2008-08-10 20:05 syslog

August 10, 2008, 06:29:38 pm

#5

mutex

mutex

Administrator
Newbie
Posts: 782

Re: individual files

I missed this:

log {
source(src);
destination(std);
};

At the very end of the file...

The problem seems to be that the expansion of $FACILITY isn't happening when the filename to write to is set.

August 10, 2008, 06:36:12 pm

#6

spider

spider

Newbie
Posts: 10

Re: individual files

Your suggesting that not be there or modified?

August 10, 2008, 06:37:27 pm

#7

mutex

mutex

Administrator
Newbie
Posts: 782

Re: individual files

thinking out loud, I suppose, while I try to figure out why it's not working.

In the single log file that is dropped into the directory, are there logs of all types?

August 10, 2008, 06:42:18 pm

#8

spider

spider

Newbie
Posts: 10

Re: individual files

actually no, it's missing information from auth for sure apears more. it's hard to tell where i dont have a whole lot of trafic to generate entries on the weekend

August 10, 2008, 06:47:41 pm

#9

spider

spider

Newbie
Posts: 10

Re: individual files

Actually scratch that. had stoped the service to make a change to the config and forgot to start it again Embarrassed

It is catching all the log info in the one file.

August 10, 2008, 06:52:07 pm

#10

mutex

mutex

Administrator
Newbie
Posts: 782

Re: individual files

To limit the number of possible variables, go ahead and comment out the filter() statements and the other log() statements up above, leaving only the log statement at the very bottom, then try restarting.

August 10, 2008, 06:56:24 pm

#11

spider

spider

Newbie
Posts: 10

Re: individual files

Yeah commenting them out made no difference. even remebered to start the service again that time lol

August 10, 2008, 07:00:57 pm

#12

mutex

mutex