# Robots.txt # --- Disallow images, photographs direcotries access ------------- # Disallow directory /App_Code/ User-agent: * Disallow: /App_Code/ # Disallow directory /App_Data/ User-agent: * Disallow: /App_Data/ # Disallow directory /Bin/ User-agent: * Disallow: /Bin/ # Disallow directory /Admin/ User-agent: * Disallow: /Admin/ # --- Disallow indexing of specific file extensions --------------- # --- Disallow all other directories access ----------------------- # --- Disallow Other robots (my list) --------------------------------- User-agent: psbot Disallow: / # --- Disallow Collectors and Spam --------------------------------- User-agent: atSpider Disallow: / User-agent: cherrypicker Disallow: / User-agent: DSurf Disallow: / User-agent: EliteSys Entry Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: Mail Sweeper Disallow: / User-agent: munky Disallow: / User-agent: Roverbot Disallow: / User-agent: WebEmailExtrac Disallow: /