Multiple destinations and droped packets

[sirius]

Hi,

I'm sending syslog records from our firewall to the syslog server running syslog-ng. I've set up syslog-ng to send the records to two destinations:

filter f_firewall   { host("192.168.200.200"); };

destination d_hosts { file("/media/syslog/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$HOST.log" owner(root) group (syslog) perm(0640) dir_group(syslog) dir_perm(0770) create_dirs(yes)); };

# Firewall
log { source(network);
      filter(f_firewall);
      destination(d_hosts);
      destination(d_mefa4);
    };

The second destination is a firewall analyser tool and the problem is that I am getting significantly fewer packets at the second destination then at the syslog server. My file on the syslog server has grown to 200mb in just and hour or so and a packet analyser captures some 20,000 packets in just a few minutes. At the second destination I receive just 200-300 packets over the same period. Monitoring the outgoing packets from the syslog server I see that just 200-300 packets are being sent to the analayser.

Is syslog-ng dropping the packets or are they being buffered somewhere? This seems to be a very large discrepancy.

[mutex]

I doubt they are being buffered.  My initial thoughts are:
1. it's a bug (that may be fixed in a later version, if you aren't already running the latest version
2. it's a load problem on the syslog-ng server

It would be very interesting to rerun your test of watching packets leaving the syslog-ng server sent to the firewall analyzer tool after commenting out the destination(d_hosts); line in the log{} statement.

[sirius]

Thanks for the reply. I've done a little more digging and it seems that this is something to do with the firewall analyzer. We are using ManageEngine's Firewall Analyser 5. I fired up the firewall analyaser and once it was up and running (takes 3-5min) did another tcpdump. The packets were streaming through!

I stopped the analyser and the tcpdump output reduced to a crawl. I started it up again and after a couple of minutes it sprung to life again. I can only assume that this weird behaviour is down to the interaction between the analyser software and the tcp/ip stack. Quite why it should throttle the traffic seen by tcpdump while it is NOT running seems rather odd.

It would seem then that at least syslog-ng is doing its job.

[mutex]

That is good to hear.  I suspect that the manage engine service stopped listening on the syslog port and each time your syslog-ng server sent a log message, it would get an ICMP packet back indicating that the port was not open on the destination host.