Block bad bots in .htaccess (copy/paste)

Posted on By Admin No Comments on Block bad bots in .htaccess (copy/paste)

Block Bad Bots in .htaccess (Copy/Paste)

If your WordPress site is suffering from unwanted traffic caused by bad bots—such as scrapers, spammers, or malicious crawlers—you can block them efficiently using your .htaccess file. This quick fix not only denies access to known bad bots but also rate limits requests to reduce server load and improve site security.

Quick Fix: Deny Bad Bots & Rate Limit in .htaccess

  1. Open your WordPress root directory and locate the .htaccess file.
  2. Backup the .htaccess file before making changes.
  3. Copy and paste the following code at the top of your .htaccess file:
# Block Bad Bots by User-Agent
SetEnvIfNoCase User-Agent "AhrefsBot" bad_bot
SetEnvIfNoCase User-Agent "SemrushBot" bad_bot
SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "DotBot" bad_bot
SetEnvIfNoCase User-Agent "BLEXBot" bad_bot
SetEnvIfNoCase User-Agent "YandexBot" bad_bot

Order Allow,Deny
Allow from all
Deny from env=bad_bot

# Rate limiting: Limit excessive requests per IP

  SetEnvIf Remote_Addr "^.*$" RATE_LIMIT
  
    SetOutputFilter RATE_LIMIT
    SetEnv rate-limit 400
  


# Alternative rate limiting using mod_reqtimeout (Apache 2.2.15+)

  RequestReadTimeout header=20-40,minrate=500
  1. Save the file and upload it back to your server if editing locally.
  2. Test your site to ensure it loads correctly and bad bots are blocked.

Why This Happens

Bad bots can cause multiple issues for WordPress sites:

  • Server overload: Excessive requests from bots can consume bandwidth and CPU, slowing down your site.
  • Security risks: Some bots scan for vulnerabilities or attempt brute force attacks.
  • SEO damage: Scrapers can steal your content, harming your search engine rankings.

Blocking bad bots at the .htaccess level prevents them from reaching your WordPress application, reducing resource usage and improving overall site performance and security.

Step-by-Step: How to Block Bad Bots in .htaccess

  1. Access your WordPress root directory: Use FTP, SFTP, or your hosting control panel’s file manager to navigate to the folder where WordPress is installed. This folder contains the wp-config.php and .htaccess files.
  2. Backup your current .htaccess file: Before making any changes, download a copy of your existing .htaccess file to your local machine. This ensures you can restore it if something goes wrong.
  3. Edit the .htaccess file: Open the .htaccess file in a plain text editor.
  4. Add bad bot blocking rules: Insert the following code at the very top of the file, before any WordPress rules:
# Block Bad Bots by User-Agent
SetEnvIfNoCase User-Agent "AhrefsBot" bad_bot
SetEnvIfNoCase User-Agent "SemrushBot" bad_bot
SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
SetEnvIfNoCase User-Agent "DotBot" bad_bot
SetEnvIfNoCase User-Agent "BLEXBot" bad_bot
SetEnvIfNoCase User-Agent "YandexBot" bad_bot

Order Allow,Deny
Allow from all
Deny from env=bad_bot

This code uses SetEnvIfNoCase to detect bad bots by their user-agent strings and denies them access.

  1. Add rate limiting rules: To prevent excessive requests from any IP address, add the following code below the bot blocking rules:
# Rate limiting: Limit excessive requests per IP

  SetEnvIf Remote_Addr "^.*$" RATE_LIMIT
  
    SetOutputFilter RATE_LIMIT
    SetEnv rate-limit 400
  


# Alternative rate limiting using mod_reqtimeout (Apache 2.2.15+)

  RequestReadTimeout header=20-40,minrate=500

This limits the bandwidth and request rate to protect your server from overload.

  1. Save and upload the file: If editing locally, upload the modified .htaccess file back to your server, overwriting the existing one.
  2. Test your website: Visit your site in a browser to ensure it loads normally. Use online tools or command line to simulate bad bot user-agents and confirm they are blocked.

Testing Your Bad Bot Blocking

  • Browser test: Your site should load normally for regular browsers.
  • Command line test: Use curl to simulate a bad bot user-agent:
curl -A "AhrefsBot" -I https://yourdomain.com/

The response should be 403 Forbidden or similar, indicating the bot is blocked.

  • Check server logs: Review your Apache error or access logs to verify that requests from bad bots are denied.

Works On

Server Software Notes
Apache Fully supported; requires mod_setenvif, mod_authz_host, and optionally mod_ratelimit or mod_reqtimeout
Nginx Does not use .htaccess; configure in server block instead
LiteSpeed Supports .htaccess rules similar to Apache
cPanel / Ples

Speed & Security Tags:, , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

Also by the maker of MySurveyReviews.com