# robots.txt for http://www-group.slac.stanford.edu/ # See documentation in: http://info.webcrawler.com/mak/projects/robots/norobots.html # Created by Joan Winters, 18 May 2000 -- Disallowed indexing of /archive/ tree # Disallowed 12 _FrontPage subdirs re Q217103 (Too bad this only disallows at root.) # Updated by Joan Winters, 7 Jun 2000 -- Disallowed indexing of /cgi-bin* trees User-agent: * Disallow: /archive/ Disallow: /cgi-bin/ Disallow: /_borders/ Disallow: /_derived/ Disallow: /_fpclass/ Disallow: /_overlay/ Disallow: /_private/ Disallow: /_themes/ Disallow: /_vti_bin/ Disallow: /_vti_cnf/ Disallow: /_vti_log/ Disallow: /_vti_map/ Disallow: /_vti_pvt/ Disallow: /_vti_txt/ Disallow: /admin/ Disallow: /exclude/ Disallow: /working/ Disallow: *cgi-bin* Disallow: /hrdev/ Disallow: /techpubs/ Disallow: /esh/rp/shared/ Disallow: /infomediadev/ Disallow: /ocfo/