# # Robot.txt file for suda.co.uk # # Brian Suda # brian@suda.co.uk # http://suda.co.uk # # Stopping the AI Bot from harvesting content # version 1.7 # 2023/10/02 # # version 1.6 # 2007/01/14 # User-agent: * Disallow: /projects/microformats/planet/ User-agent: Exabot Disallow: * User-agent: msnbot Disallow: * User-agent: Google-Extended Disallow: / # =========================================================== # honeypot items # monitor these items if they show-up in the 404 error report # the following folders do NOT exists, # any robot that these links are farming for something Disallow: /email # honeypot trap Disallow: /e-mail # honeypot trap Disallow: /mail # honeypot trap Disallow: /address # honeypot trap Disallow: /personal # honeypot trap