Google Search will no longer make site backups while crawling the web.
  • Smoke@beehaw.orgEnglish
    3·
    2 年前

    There’s ways to rate limit, like increasing response time per IP address per hour to make rapid, massed requests slower and easier to handle. Taking them all down at once is an extreme move.