Rotating log file specified in log source

[kalkotivinay]

Hi,

I have a program which reads logs from the unixstream(). Syslog-ng reads logs from a file and send them to this unixstream.

So, I have added the log file in the source{} and unixstream in the destination{}.

As time goes, the log file is getting bigger and the plan to is rotate it. How should I make sure that syslog-ng reads all the events from this file before the file is rotated ?.

Thanks,
Vinay

[mutex]

Is your concern that when you send a HUP signal to syslog-ng that it'll lose data if there is stuff in the stream waiting to be read?  I think it will block until it's read.

[kalkotivinay]

I have fixed the HUP signal issue. The problem was in my code. It was nothing related to syslog-ng.

I have a situation where syslog-ng is reading from a file (myprocess.log) and writing the events to the unixstream. Another process is reading the events from the unixstream.

The process which writes to myprocess.log is a daemon. So, the plan is to logrotate the myprocess.log file. Since syslog-ng is reading events from the myprocess.log file, my concern is that syslog-ng might miss reading some events when the myprocess.log is rotated.

My question is will I run into this problem and is there a solution for this ?

[mutex]

I understand now.  That's a good question.  How are you reading the logs with syslog-ng?  using the source file() directive? 

[kalkotivinay]

my source looks like this -->
source unixstream_src { file("/var/log/myprocess.log" follow_freq(1) flags(no-parse)); };

[Admin]

I suspect it could happen.  Do you have follow_freq(1) set for a reason?  I think it's much less likely to happen without follow_freq(1), but then if you have it set for a specific reason, it's a moot point.

The other way to do this, though it's a bit of a hack, is this:

Code:

tail -f /var/log/myprocess.log | logger -h localhost

Of course, you would have to set up syslog-ng to listen on the network, and handle restarting the tail after a log rotate, unless you copy the log, then clear - something like this:

Code:

cp /var/log/myprocess.log /var/log/myprocess.log.0
echo '' >/var/log/myprocess.log

[kalkotivinay]

The reason why I have specified follow_freq(1) is that on syslog-ng startup,  /var/log/myprocess.log file might not be created and when syslog-ng starts, it throw an error. So, I have specified the follow_freq(1).

[Admin]

Ok.  I am not certain of the polling interval when follow_freq is on.  I think you are right to be concerned about losing logs.  If it is critical that you don't miss any logs, you may have to resort to something like the tail suggestion above.