User-agent: * Disallow: /apps/ Disallow: /*.xls$ Disallow: /*.doc$ Disallow: /*.pdf$ Disallow: /NR/ # ALLOW LOCAL GOOGLE MINI APPLIANCE CRAWLER (disallow nothing) User-agent: gsa-crawler-ppai Disallow: