Virtual Architects Support Wiki

User Tools

Site Tools

You are here: start » networking » linux » bad_bot_trap
Trace: • ftp_server
networking:linux:bad_bot_trap

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
networking:linux:bad_bot_trap [2013/10/13 14:50]
gcooper 		networking:linux:bad_bot_trap [2013/10/13 14:59] (current)
gcooper
Line 8: Line 8:
  
 ===== robots.txt ===== ===== robots.txt =====
 +
 +Misbehaving bots may access areas of your web site even if you tell then not to.  That's what we key on here.
  
 :!: First, change to your web root folder.  Possibly ''public_html''. :!: First, change to your web root folder.  Possibly ''public_html''.
Line 21: Line 23:
 Disallow: /bot-trap/ Disallow: /bot-trap/
 </file> </file>
 +
 +===== Web Site Header =====
  
 We use a tiny image for embedding the hidden link.  ''blank.png'' should be just a single pixel transparent image and is unimportant (and may already exist).  If it doesn't already exist, you can get ''blank.png'' like this: We use a tiny image for embedding the hidden link.  ''blank.png'' should be just a single pixel transparent image and is unimportant (and may already exist).  If it doesn't already exist, you can get ''blank.png'' like this:
Line 30: Line 34:
 </file> </file>
  
-Edit your HTML header (''<head>'' section) (''index.html'', ''templates/yourtemplatename/index.php'', etc.) and add this line:+Edit your HTML header (''<head>'' section) (''index.html'', ''templates/yourtemplatename/index.php'', etc.) and add this line (modify as necessary):
  
 <file> <file>
Line 37: Line 41:
 <a href="/bot-trap/"><img src="images/blank.png" border="0" alt=" " width="1" height="1"></a> <a href="/bot-trap/"><img src="images/blank.png" border="0" alt=" " width="1" height="1"></a>
 </file> </file>
 +
 +===== bot-trap Folder =====
  
 Now we create the ''bot-trap'' folder and a file so as not to pollute the error logs.  Change the URL to your own domain and the permissions as necessary: Now we create the ''bot-trap'' folder and a file so as not to pollute the error logs.  Change the URL to your own domain and the permissions as necessary:
Line 54: Line 60:
 chown -R apache.apache bot-trap chown -R apache.apache bot-trap
 </file> </file>
 +
 +===== Fail2Ban =====
  
 Add another regex to the fail2ban ''badbots'' filter: Add another regex to the fail2ban ''badbots'' filter:
Line 69: Line 77:
 service fail2ban restart service fail2ban restart
 </file> </file>
 +
 +===== Test Fail2Ban Filter =====
 +
 +Modify your log path as necessary:
 +
 +<file>
 +fail2ban-regex ../logs/access_log /etc/fail2ban/filter.d/apache-badbots.conf
 +</file>
 +
 +Check the Fail2Ban log:
 +
 +<file>
 +tail -f /var/log/fail2ban.log
 +</file>
 +
 +:!: If Fail2Ban fails to parse your log files at all, try setting ''backend=polling'' in ''jail.local''.
  
 ===== Parse IPTables Rules for List of Banned IPs ===== ===== Parse IPTables Rules for List of Banned IPs =====
networking/linux/bad_bot_trap.1381697415.txt.gz · Last modified: 2013/10/13 14:50 by gcooper

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki