I recently wanted to block all bots visiting http://example.com/view/1.php, http://example.com/view/2.php but only allow Google Adsense bot. You need to edit robots.txt (a text file prevent cooperating web spiders and other web robots from accessing all or part of a website) as follows
User-agent: Mediapartners-Google Allow: / User-agent: * Disallow: /view/*.php
{ 2 comments… read them below or add one }
With ht.access is it possible also
thnx man… i really need this script.