The free internet encyclopedia is the seventh-most visited website in the world, and it wants to stay that way.
  • pivot_root@lemmy.world
    8·
    4 months ago

    “robots.txt files? You mean those things we use as part of the site index when scraping it?”

    — AI companies, probably