# /robots.txt file for http://webcrawler.com/ # mail webmaster@webcrawler.com for constructive criticism User-agent: webcrawler Disallow: /stat User-agent: lycra Disallow: /stat User-agent: * Disallow: /stat Disallow: /content Disallow: /FOTO Disallow: /imagevue Disallow: /mp3 Disallow: /content Disallow: /OLD Disallow: /press Disallow: /test Disallow: /content Disallow: /imagevue Disallow: /IMGV Disallow: /pannelli Disallow: /secure Disallow: /res Disallow: /press Disallow: /cgi-bin Disallow: /logs Disallow: /hotel Disallow: /mostra Disallow: /city-spaces Disallow: /distruction Disallow: /emotion Disallow: /top_files Disallow: /FORUM Disallow: /foRum Disallow: /forum Disallow: /rogito Disallow: /unpredictable Disallow: /workers Disallow: /script Disallow: /www1 Disallow: /icon Disallow: /SKIN Disallow: /varie-foto Disallow: /silence Disallow: /filmati Disallow: /jasmine Disallow: /gaia Disallow: /ritratti Disallow: /top-files Disallow: /software Disallow: /stili MG Disallow: /testing.marcogiardini Disallow: /mercato Disallow: /PDF Disallow: /myanmar Disallow: /res Disallow: /robots.txt # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: / # Disallow: /