User Tools
networking:linux:bad_bot_trap
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
networking:linux:bad_bot_trap [2013/09/26 11:13] gcooper |
networking:linux:bad_bot_trap [2013/10/13 14:59] (current) gcooper |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Bad Bot Trap ====== | ====== Bad Bot Trap ====== | ||
| + | |||
| + | See also **[[networking: | ||
| http:// | http:// | ||
| - | Used here with **[[networking: | + | We use Fail2Ban to block bad bots. |
| + | |||
| + | ===== robots.txt ===== | ||
| + | |||
| + | Misbehaving bots may access areas of your web site even if you tell then not to. That's what we key on here. | ||
| :!: First, change to your web root folder. | :!: First, change to your web root folder. | ||
| Line 18: | Line 24: | ||
| </ | </ | ||
| - | < | + | ===== Web Site Header ===== |
| - | mkdir bot-trap | + | |
| - | </ | + | |
| - | '' | + | We use a tiny image for embedding the hidden link. |
| < | < | ||
| Line 30: | Line 34: | ||
| </ | </ | ||
| - | Edit your HTML header (''< | + | Edit your HTML header (''< |
| < | < | ||
| Line 38: | Line 42: | ||
| </ | </ | ||
| - | Now we create a file so as not to pollute the error logs: | + | ===== bot-trap Folder ===== |
| + | |||
| + | Now we create | ||
| < | < | ||
| + | mkdir bot-trap | ||
| cat << EOF >> bot-trap/ | cat << EOF >> bot-trap/ | ||
| < | < | ||
| Line 49: | Line 56: | ||
| </ | </ | ||
| </ | </ | ||
| - | } | ||
| EOF | EOF | ||
| + | |||
| + | chown -R apache.apache bot-trap | ||
| </ | </ | ||
| - | Add another regex to the fail2ban filter: | + | ===== Fail2Ban ===== |
| + | |||
| + | Add another regex to the fail2ban | ||
| < | < | ||
| + | vi / | ||
| + | |||
| failregex = ^< | failregex = ^< | ||
| ^< | ^< | ||
| + | </ | ||
| + | |||
| + | Be sure to enable the '' | ||
| + | |||
| + | < | ||
| + | service fail2ban restart | ||
| + | </ | ||
| + | |||
| + | ===== Test Fail2Ban Filter ===== | ||
| + | |||
| + | Modify your log path as necessary: | ||
| + | |||
| + | < | ||
| + | fail2ban-regex ../ | ||
| + | </ | ||
| + | |||
| + | Check the Fail2Ban log: | ||
| + | |||
| + | < | ||
| + | tail -f / | ||
| + | </ | ||
| + | |||
| + | :!: If Fail2Ban fails to parse your log files at all, try setting '' | ||
| + | |||
| + | ===== Parse IPTables Rules for List of Banned IPs ===== | ||
| + | |||
| + | < | ||
| + | iptables -nL |grep " | ||
| </ | </ | ||
networking/linux/bad_bot_trap.1380215636.txt.gz · Last modified: 2013/09/26 11:13 by gcooper
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
