podivne zpomaleni provozu s firewallem PF

[Miroslav Lachman]

Na testovacim stroji v lokalni siti jsem se rozhodl otestovat FreeBSD 7, 
takze jsem si nejprve chtel udelat benchmark MySQL a Apache se soucasnou 
verzi 6.2 a pak porovnat stejne testy na FreeBSD 7.0-BETA3. Pri tomto 
testovani jsem narazil na podivne chovani pri testu Apache. Ten jsem 
testoval z druheho stroje:

ab -c 15 -n 50000 http://192.168.1.164/phpinfo.php

Test se choval dost nestabilne, chvili jel rychle, pak zpomalil, pak 
zase zrychlil. Nejprve jsem to sledoval tailem v access logu Apache, ale 
pak jsem se rozhodnul vypnout logovani, abych vyloucil jeste nejake 
zdrzovani se zapisem na disk. Takze jsem se na provoz parkrat kouknul 
tcpdumpem a tam se deje nasledujici vec - provoz jede priblizne 20-25 
sekund, pak na chvili ustane (zhruba 5 sekund), pak se zase na 5-8 
sekund rozjede, po te opet ustane na nejakych 5 sekund a opet se rozjede 
na priblizne 20-25 sekund a takhle se to porad dokola opakuje. V 
okamziku, kdy provoz ustane, je CPU nevytizeny (temer 100% idle) a 
system normalne reaguje, takze to nevypada na nejake celkove zamrzavani 
systemu, jen toho TCP provozu smerem na Apache.

## top zachyceny uprostred benchmarku v okamziku "pauzy"
last pid:  2182;  load averages: 10.08,  8.79,  7.59         up 
0+01:18:29  12:32:06
64 processes:  16 running, 48 sleeping
CPU states:  3.4% user,  0.0% nice,  2.3% system,  3.0% interrupt, 91.4% 
idle
Mem: 136M Active, 25M Inact, 56M Wired, 10M Cache, 34M Buf, 15M Free
Swap: 512M Total, 512M Free

Pokud vypnu PF (pfctl -d), tak k tomu nedochazi a vse jede od zacatku az 
do konce plynule. Rozdil v benchmarku se zapnutym a vypnutym PF je 
znacny = 90 req/s vs. 160 req/s

## top zachyceny pri vypnutem PF
last pid:  1809;  load averages: 13.79,  9.78,  7.61         up 
0+00:54:13  12:07:50
68 processes:  15 running, 53 sleeping
CPU states: 62.0% user,  0.0% nice, 13.5% system, 24.4% interrupt,  0.0% 
idle
Mem: 140M Active, 33M Inact, 56M Wired, 11M Cache, 34M Buf, 2652K Free
Swap: 512M Total, 512M Free

Z meho lajckeho pohledu hadam, ze to bude mit mozna neco spolecneho s 
kontrolou state, nebo znovupouzivanim odchozich portu, ale na tehle 
urovni tomu uz bohuzel vubec nerozumim.

Dokazal by me nekdo nasmerovat k nejakemu "tuningu" toho PF, aby se 
tohle nedelo, nebo dokaze aspon nekdo zodpovedne posoudit, jestli je to 
zkratka "nevyhnutelne" pri pouziti PF?

Pro uplnost prikladam i pouzita pravidla pf.conf (zatim jsem je 
nezkousel upravovat, abych zjistil, ktera cast na to ma vliv, ale urcite 
to zkusim, jakmile na to najdu trochu casu)

ext_if="vr0"
ext_addr_0="192.168.1.164"
ext_tcp_0_inports="{ 21, 25, 80, 110, 143, 443, 465, 587, 993, 995 }"
ext_ssh_0="22"
# secondary IPs of ext. interface - allowing public services
ext_addr_1="192.168.1.165"
ext_tcp_1_inports="{ 21, 22, 25, 80, 110, 143, 443, 587, 993, 995 }"
ext_addr_2="192.168.1.166"
ext_tcp_2_inports="{ 21, 22, 25, 80, 443 }"

jail_addr_0="10.11.12.13"
jail_tcp_0_inports="{ 21, 22, 25, 80, 443 }"
jail_addr_1="10.11.12.14"
jail_tcp_1_inports="{ 21, 22, 25, 80, 443 }"

samba_ports_udp="{ netbios-ns netbios-dgm }"
samba_ports_tcp="{ netbios-ssn microsoft-ds }"

unfiltered="{ lo0, lo1 }"

table <reserved> { 172.16.0.0/12, 10.0.0.0/8, 127.0.0.0/8, 0.0.0.0/8, 
169.254.0.0/16, 192.0.2.0/24, 204.152.64.0/23, 224.0.0.0/3, ! 
$jail_addr_0, ! $jail_addr_1 }
table <czech_net> persist file "/etc/pf.czech_net.table"
table <goodguys> persist file "/etc/pf.goodguys.table"
table <badguys> persist file "/etc/pf.badguys.table"
table <bruteforce> persist
table <ssh_bruteforce> persist
table <lan> { 192.168.1.0/24 }

set timeout { interval 10, frag 20 }
set limit { states 10000, frags 5000 }
set optimization aggressive
set block-policy return

# Let loopback and internal interface traffic flow without restrictions
set skip on $unfiltered

scrub in on $ext_if
scrub out on $ext_if no-df random-id min-ttl 24 max-mss 1492

binat on $ext_if from $jail_addr_0 to any -> $ext_addr_1
binat on $ext_if from $jail_addr_1 to any -> $ext_addr_2

pass in quick proto tcp from <goodguys> to any port $ext_ssh_0 flags 
S/SA keep state

# deny bad addresses from tables
block in quick from { <badguys>, <bruteforce>, <ssh_bruteforce> } to any

block quick inet6 all
block

block quick on $ext_if inet from <reserved> to any
block quick on $ext_if inet from any to <reserved>

antispoof quick for { $ext_if, lo0 }
# Allow SSH connections from czech IPs
pass in log on $ext_if proto tcp from <czech_net> to $ext_addr_0 port 
$ext_ssh_0 flags S/SA keep state \
         (max-src-conn 6, max-src-conn-rate 6/60, overload 
<ssh_bruteforce> flush global)

pass out quick on $ext_if inet proto icmp icmp-type 8 code 0 keep state
pass in quick on $ext_if inet proto icmp icmp-type 8 code 0 keep state

pass out on $ext_if inet proto udp keep state

pass out on $ext_if inet proto tcp from $ext_if to any flags S/SA 
modulate state
pass in on $ext_if inet proto tcp from any to $ext_addr_0 port 
$ext_tcp_0_inports flags S/SA keep state

pass in on $ext_if inet proto tcp from any to $jail_addr_0 port 
$jail_tcp_0_inports flags S/SA keep state
pass in on $ext_if inet proto tcp from any to $jail_addr_1 port 
$jail_tcp_1_inports flags S/SA keep state

pass in quick on $ext_if proto tcp from <lan> to $ext_addr_0 port 
$samba_ports_tcp modulate state
pass in quick on $ext_if proto udp from <lan> to $ext_addr_0 port 
$samba_ports_udp keep state

pass in on $ext_if inet proto tcp from any to $ext_addr_0 port 55000 >< 
56000 keep state


Pro uplnost jeste vysvetlim, ze tabulky badguys, bruteforce, 
ssh_bruteforce jsou v tomto pripade prazdne a tabulka czech_net obsahuje 
rozsahy "vsech" ceskych IP adres vytazenych z Geo IP databaze.

Jaily maji sve vlastni privatni IP adresy na zarizeni lo1 a je na ne 
udelana binat.

Mirek