OpenBSD Journal

Hatchet-0.8 (PF Log Parser ) for OpenBSD released

Contributed by sean on from the when you must have your pie chart dept.

Dan writes:
Jason Dixon announces the release of Hatchet 0.8 - a PF log parser and web interface. He says this is a fairly signifcant release with bug fixes and focus on supporting the default OpenBSD httpd chroot "out of the box".

Hatchet is a log parsing/presentation program written for OpenBSD's PF logs. Hatchet uses a series of Perl regexes to match entries from the pflog logs. The log entries are stored in a SQLite database file, allowing for highly dynamic queries and statistics. Jason says that Hatchet should work fine with FreeBSD 5.x or NetBSD 2.x with the PF-enabled tcpdump.

(Comments are closed)

Comments

  1. By Mr.Pantz (204.94.49.143) on http://www.pantz.org

    I checked out a few PF log stats programs in the past. None gave me really what I wanted. All seemed like overkill or just not enough in other areas. I just wanted to see what ports attackers and scanners were hitting. And what IP's they were from. Mostly based on my default block policy.

    So with my terrible Perl skills I wipped up a script that did just what I wanted. Shows top ports blocked and top IP's blocked. Then breaks it down with counts of ports per IP blocked and IP's per port blocked. It all dumps to an HTML file with simple tables.

    If anyone is intrested its at:

    http://www.pantz.org/os/openbsd/pantzpfstats.shtml

    p.s - This was just modified to work with 3.7 as the log format changed ever so slightly.

    Comments

    1. By Anonymous Coward (216.238.113.174) on

      One of the things I would like from pf (or a pf log parser) is the ability to generate a report similar to the following ipfw report on FreeBSD (note: IP addresses have been munged)

      Columns are: rule number, number of hits, number of bytes, rule text 

      
# ipfw show 
00100  5834   817100 allow ip from any to any via lo0
00101     0        0 check-state
00101     0        0 allow tcp from 10.119.50.0/10 to me 23 keep-state
00102 21067 14948739 allow ip from me to any keep-state
00102     0        0 allow tcp from 10.238.113.0/10 to me 23 keep-state
00105  2539   321862 allow tcp from 10.119.50.0/10 to me 73 keep-state
00106  2100   282575 allow tcp from 10.238.113.0/10 to me 73 keep-state
00107     0        0 allow tcp from 10.151.11.86 to me 73 keep-state
00108     0        0 allow tcp from 10.109.220.101 to me 73 keep-state
00109     0        0 allow log logamount 10 tcp from 10.234.161.11 to me 73 keep-state
00110     0        0 allow log logamount 10 tcp from 10.209.57.0/10 to me 73 keep-state
00111     0        0 allow log logamount 10 tcp from 10.177.155.0/10 to me 73 keep-state
00121     0        0 allow log logamount 10 tcp from 10.0.0.0/10 to me 73 keep-state
00122     0        0 allow log logamount 10 tcp from 10.0.0.0/10 to me 75 keep-state
00155     0        0 allow tcp from 10.119.50.0/10 to me 75 keep-state
00156     0        0 allow tcp from 10.238.113.0/10 to me 75 keep-state
00157     0        0 allow tcp from 10.151.11.86 to me 75 keep-state
00158     0        0 allow tcp from 10.109.220.101 to me 75 keep-state
00159     0        0 allow log logamount 10 tcp from 10.234.161.11 to me 75 keep-state
00160     0        0 allow log logamount 10 tcp from 10.209.57.0/10 to me 75 keep-state
00161     0        0 allow log logamount 10 tcp from 10.177.155.0/10 to me 75 keep-state
00220  1186    67512 allow tcp from any to 10.194.67.64 80 keep-state
00221 21016 17065272 allow tcp from any to 10.51.122.145 80 keep-state
00222   171    61101 allow tcp from any to 10.51.122.146 80 keep-state
00223   192    55385 allow tcp from any to 10.51.122.147 80 keep-state
00210   125    41315 allow tcp from any to 10.51.122.148 80 keep-state
00225 18488 14295438 allow tcp from any to 10.51.122.149 80 keep-state
00226    30     1512 deny tcp from any to 10.51.122.150 80 keep-state
00227    13      610 deny tcp from any to 10.51.122.151 80 keep-state
00228    13      616 deny tcp from any to 10.51.122.152 80 keep-state
00229    15      720 deny tcp from any to 10.51.122.153 80 keep-state
00230    32     1510 deny tcp from any to 10.51.122.154 80 keep-state
00231   210    80336 allow tcp from any to 10.51.122.155 80 keep-state
00232   184    65705 allow tcp from any to 10.51.122.156 80 keep-state
00233    26     1351 deny tcp from any to 10.51.122.157 80 keep-state
00234    18      891 deny tcp from any to 10.51.122.158 80 keep-state
02101     6      360 unreach host tcp from 8.10.161.9 to me 25
02201     7      336 unreach host tcp from 61.0.0.0/8 to me 25
02206     0        0 unreach host tcp from 10.232.128.0/19 to me 25
02211     0        0 unreach host tcp from 10.109.10.0/21 to me 25
0210     0        0 unreach host tcp from 10.1.192.0/19 to me 25
02221     0        0 unreach host tcp from 10.44.56.0/21 to me 25
02226     0        0 unreach host tcp from 10.6.10.0/10 to me 25
02231     0        0 unreach host tcp from 110.226.0.0/16 to me 25
02233    13      610 unreach host tcp from 200.0.0.0/8 to me 25
02234     6      288 unreach host tcp from 202.0.0.0/7 to me 25
02235     0        0 unreach host tcp from 209.42.32.0/19 to me 25
02236    11      528 unreach host tcp from 210.0.0.0/7 to me 25
02101     0        0 unreach host tcp from 10.33.86.0/10 to me 25
02106     9      432 unreach host tcp from 218.0.0.0/8 to me 25
02251     6      288 unreach host tcp from 219.0.0.0/8 to me 25
02256    20     1200 unreach host tcp from 220.0.0.0/8 to me 25
02261     0        0 unreach host tcp from 221.0.0.0/8 to me 25
02210    45     100 unreach host tcp from 222.0.0.0/8 to me 25
02304 23209 14578964 allow tcp from any to me 25 keep-state
02308 17878 1310107 allow tcp from any to me 995 keep-state
02355  1810   827077 allow tcp from 10.119.50.0/10 to me 74 keep-state
02356    88     6554 allow tcp from 10.238.113.0/10 to me 74 keep-state
02357     0        0 allow log logamount 10 tcp from 10.0.0.0/10 to me 74 keep-state
02506   936   101832 allow tcp from any to me 443 keep-state
02605     0        0 allow tcp from 10.0.0.0/10 to me 71 keep-state
02606  5881   417472 allow tcp from 10.238.113.0/10 to me 71 keep-state
02606     0        0 allow log logamount 10 tcp from any to me 71 keep-state
02606    21     1092 allow log logamount 10 tcp from any to me 20 keep-state
02607     0        0 allow tcp from 10.0.0.0/10 to me 20 keep-state
02607     0        0 allow tcp from 10.238.113.0/10 to me 20 keep-state
02608     0        0 allow tcp from 10.119.50.193 to me 71 keep-state
02609     0        0 allow tcp from 10.119.50.193 to me 20 keep-state
02610     0        0 allow tcp from 10.126.41.236 to me 71 keep-state
02611     0        0 allow tcp from 10.126.41.236 to me 20 keep-state
02612     0        0 allow tcp from 64.252.50.47 to me 71 keep-state
02613     0        0 allow tcp from 64.252.50.47 to me 20 keep-state
54001   555    17474 allow icmp from any to me icmptype 8
55000     2      112 allow icmp from any to me icmptype 11
60000    14      840 deny tcp from any to me 21
60001    73     4310 deny tcp from any to me 22
60002     0        0 deny tcp from any to me 23
60003     0        0 deny tcp from any to me 53
60004     0        0 deny tcp from any to me 110
60006     0        0 deny tcp from any to me 111
60007   611    29716 deny tcp from any to me 135
60008     0        0 deny tcp from any to me 137
60010   851    40880 deny tcp from any to me 139
60012  1092    53052 deny tcp from any to me 445
60014     0        0 deny tcp from any to me 515
60016     0        0 deny tcp from any to me 554
60020     1       40 deny tcp from any to me 1080
60021     0        0 deny tcp from any to me 1197
60022     0        0 deny tcp from any to me 110
60010    61     2808 deny tcp from any to me 1433
60026     0        0 deny tcp from any to me 1443
60027     3      136 deny tcp from any to me 3128
60028     0        0 deny tcp from any to me 3389
60029     0        0 deny tcp from any to me 4444
60030     0        0 deny tcp from any to me 7955
60031     3      136 deny tcp from any to me 8080
60032     0        0 deny tcp from any to me 11831
60033     0        0 deny tcp from any to me 12345
60034     0        0 deny tcp from any to me 17300
60035     0        0 deny tcp from any to me 27347
60036     0        0 deny tcp from any to me 27374
61003     0        0 deny udp from any 53 to me
61008     8      320 deny tcp from any 80 to me
61500     0        0 deny udp from any to me 53
62000     0        0 deny udp from any to any 67,10
62002     0        0 deny udp from any to any 135
62003     0        0 deny udp from any to any 136
62004 11271   879174 deny log logamount 1000 udp from any to any 137
62005  5371  1239965 deny udp from any to any 138
62006     0        0 deny udp from any to any 139
62009     0        0 deny udp from any to any 513
62010     0        0 deny udp from any to any 525
62011     0        0 deny udp from any to any 5882
62050     0        0 deny udp from any to any 31789
62150     8      322 deny udp from any to 255.255.255.255
10100   259    23160 deny ip from any to 10.51.122.144
10101   233    21535 deny ip from any to 10.51.122.159
64000     1       88 deny icmp from any to me
65000   314    15456 deny log logamount 1000 tcp from any to me
65001     0        0 deny log logamount 1000 tcp from me to any
65002   375   213102 deny log logamount 1000 udp from any to me
65003     0        0 deny log logamount 1000 udp from me to any
65500     0        0 deny log logamount 1000 ip from any to any
65535     1       78 deny ip from any to any

      Comments

      1. By Anonymous Coward (69.110.155.206) on

        pfctl -vv -s rules

        Comments

        1. By Anonymous Coward (63.119.50.193) on

          Nope, that's not it. The command you offer is way too verbose, I tried that one a year or so ago. The ipfw command produces nice, compact output.

          Comments

          1. By Anonymous Coward (69.197.92.181) on

            Yes, that is it. Its just a different format. If you don't like the format, then pipe it to awk and format it however you like.

          2. By m0rf (68.104.57.241) on

            | awk '{ruleno=$1; ruletext=$0; sub(ruleno " ", "", ruletext); sub("@", "", ruleno); getline; rulehits=$3;rulebytes=$7;getline;print ruleno " " rulehits " " rulebytes " " ruletext}'

            horribly ugly, but works, doesn't check that it reads the right fields or anything. fixing it is left as an exercise for the reader.

            Comments

            1. By Anonymous Coward (68.104.57.241) on

              doesn't output it in an aligned way either *shrug*

              Comments

              1. By Bert (216.175.250.42) on

                Don't you think that's why the person who was kind enough to hand you this gave the following caveat:

                horribly ugly, but works, doesn't check that it reads the right fields or anything. fixing it is left as an exercise for the reader

                Your brain needs exercise just as much as your pecs.

                Comments

                1. By Anonymous Coward (216.175.250.42) on

                  please ignore previous...saw the AC response, assumed that it was someone picking nits...then checked the IP.

                  Man, giving up caffeine is hell...

  2. By B.S.D Al (64.230.18.164) openbsd@otterhole.ca on http://www.otterhole.ca/pfrtg/

    I spent some time writing up a report for PF logs.

    The link is included, pfrtg, but be careful, my web site is, er, old and near useless.

    Output looks something like this: 

    Date: 2005/05/13 00:00:48 D - 2005/05/13 11:47:23 D Eastern
Source and Port Plus  (1180680 bytes  8716 packets)
 count:          source 45678901n12345678901M123G  bytes  dstn ports
 -----:          ------ 012345678901n12345678901E ------  ----  -----
  1500: 0.0.0.0         C...C...C................  72000 252 N 1:  139
   314: 204.60.78.55    ...C.....................  14482 206 N 1:  135
   265: 218.66.104.133  .......C.................  93810 174 N 2:  1026
   264: 222.77.185.228  ....XC9.................. 115896 179 N 2:  1026
    This shows the top packet hit counts to my home, when the packets came my way by the hour, the source, number of different destination addresses in my class C, and the ports hit. Perhaps too simple, and the code too ugly, but the output seems more useful than anything else I've seen.

Latest Articles

Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]