I wondered about the robots.txt

I can see the case for it, I could also see the case for allowing at least Google to index the site.

Has there been some discussion about this previously?