Example Session with Log-Templater

1 BIND 9 Logging-Configuration for Artificial Ignorance

For AI log analysis we don't want to have the time-stamp or severity in the log messages.

Below is a BIND 9 configuration fragment for an AI log file. It is recommended to write the same log messages also into other file or syslog channels: 

  channel ai  {
     file "ai-bind.log"
     versions 1
     size 100M;
     print-time no;
     print-severity no;
     severity warning;
  };

  category default        { default_syslog;  named; ai; };
  category general        { default_syslog;  named; ai; };
  category security       { security; ai; };
  category dnssec         { dnssec; ai; };
  category database       { capacity; };
  category client         { default_syslog; named; ai; };
  category network        { default_syslog; named; ai; };

2 Installation of Log-Templater on Debian Linux

mkdir ~/src
cd ~/src
apt install git automake autoconf build-essential
git clone https://github.com/rondilley/tmpltr.git
cd tmpltr
./bootstrap
autoreconf -i
./configure
make
make install
export PATH=$PATH:/usr/local/bin

3 Using Log-Templater

  • Analyse the BIND 9 "ai" log file with Log-Templater (this shell pipeline will print the top 10 log message types, including the message text of the first message of its type) 
# cat /var/cache/bind/ai-bind.log | tmpltr - | sort -rn |  sed -e 's/%s.*||//' | head
Opening [-] for read
          90 automatic empty zone: 16.172.IN-ADDR.ARPA
           7 GeoIP Region (type 3) DB not available
           6 zone 0.in-addr.arpa/IN: loaded serial 1
           5 all zones loaded
           5 stopping command channel on 127.0.0.1#953
           4 running
           4 zone localhost/IN: loaded serial 2
           4 GeoIP City (IPv4) (type 2) DB not available
           3 managed-keys-zone: Key 20326 for zone . acceptance timer complete: key now trusted
           3 automatic empty zone: 8.E.F.IP6.ARPA

3.1 Creating an ignore file

  • Next we create an ignore file. By default, this ignore file will contain all the message patterns that will be ignored in the next run. The source log file should not have any log messages that are important and should not be ignored! 
cat /var/cache/bind/ai-bind.log | tmpltr -w /var/cache/bind/bind9.ignore -

3.2 Testing the ignore file

  • The next run using the ignore file should not report any new log messages 
cat /var/cache/bind/ai-bind.log | tmpltr -t /var/cache/bind/bind9.ignore - | sort -n |  sed -e 's/%s.*||//'

3.3 New log events

  • To test the Log-Templater, we create a new event (an error message in the log) 
# rndc refresh isc.org
rndc: 'refresh' failed: not found
no matching zone 'isc.org' in any view

3.4 Check for new, never before seen log-messages

  • Periodically, we check for new, previously unseen log message types 
# cat /var/cache/bind/ai-bind.log | tmpltr -t /var/cache/bind/bind9.ignore - | sort -n |  sed -e 's/%s.*||//'
Opening [-] for read
           2 received control channel command 'refresh isc.org'
  • Now, if this is an important log message type, fix the root cause of the message
    • else create a new ignore file containing a pattern that filters out this type of message