User-agent: * Disallow: /administrator/ Disallow: /cache/ Disallow: /components/ Disallow: /editor/ Disallow: /help/ Disallow: /images/ Disallow: /includes/ Disallow: /language/ Disallow: /mambots/ Disallow: /media/ Disallow: /modules/ Disallow: /templates/ Disallow: /installation/ User-agent: Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; info@szukacz.pl) Disallow: / User-agent: LWP::Simple/5.803 Disallow: / User-agent: appie 1.1 (www.walhello.com) Disallow: / User-agent: http://www.almaden.ibm.com/cs/crawler [fc12] Disallow: / User-agent: pipeLiner/0.7 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html; webmaster@pipeline-search.com) Disallow: / User-agent: fukuiben (http://www.ibgakuin.ac.jp/~net/java/tes/fukui1.cgi) Disallow: / User-agent: sohu-search Disallow: / User-agent: DELTASCAN Disallow: / User-agent: ZoomSpider - wrensoft.com Disallow: / User-agent: SurveyBot/2.3 Disallow: / User-agent: ichiro/1.0 (ichiro@nttr.co.jp) Disallow: / User-agent: ShowTags/1.0 libwww/5.4.0 Disallow: / User-agent: CydralSpider/1.9 (Cydral Web Image Search; http://www.cydral.com) Disallow: / User-agent: Holmes/1.0 Disallow: / User-agent: UP.Browser/6.1.0.1.140 (Google CHTML Proxy/1.0) Disallow: / User-agent: Mozilla/5.0 (compatible; BecomeBot/2.2.1; MSIE 6.0 compatible; +http://www.become.com/webmasters.html) Disallow: / User-agent: IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler) Disallow: / User-agent: OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Disallow: / User-agent: EasyDL/3.04 (http://keywen.com/Encyclopedia/Bot) Disallow: / User-agent: WebCopier v3.0 Disallow: / User-agent: NetResearchServer (NRS) Disallow: / User-agent: LmCrawler Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows XP Professional Bot v.5.) Disallow: / User-agent: Missigua Locator 1.9 Disallow: / User-agent: TurnitinBot/2.0 Disallow: / User-agent: boitho.com-dc/0.71 Disallow: / User-agent: NMG Spider/0.3 (szukanko.com) Disallow: / User-agent: NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) Disallow: / User-agent: PlantyNet_WebRobot_V1.9 dhkang@plantynet.com Disallow: / User-agent: NP/0.1 (NP; http://www.nameprotect.com; npbot@nameprotect.com) Disallow: / User-agent: Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker (wn.dlc@looksmart.net; ]http://www.WISEnutbot.com) Disallow: / User-agent: e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) Disallow: / User-agent: OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Internet Categorizer Disallow: / User-agent: Vagabondo/3.0 (webagent at wise-guys dot nl) Disallow: / User-agent: Mozilla/4.0 (compatible; [b]Vagabondo/2.3; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) Disallow: / User-agent: Norbert the Spider(Burf.com) Disallow: / User-agent: larbin_2.6.3 larbin2.6.3@unspecified.mail Disallow: / User-agent: ObjectsSearch/0.06 Disallow: / User-agent: Jakarta Commons-HttpClient/2.0M1 Disallow: / User-agent: psbot/0.1 (+http://www.picsearch.com/bot.html) Disallow: / User-agent: krawler/0.1alfa Disallow: / User-agent: Zongbot /1.0 (+http://www.zong.pl/addurl.php) Disallow: / User-agent: NG/2.0 Disallow: / User-agent: OmniExplorer_Bot Disallow: /