Helo guys ive been going mad trying to get around this one so now i put it in the php forum since i heard there is a php solution to this i just cant find it online.

Im being crawled by other websites and it nailing my bandwidth i suspect they are crawling me for my backlink profile. I have tried disalow in the robot.txt file which doesn't seem to work .
Also i have used mod rewrite;s on my Apache servers htacesss file specifically blocking my user agent string and ip ranges. However i have come to find that the suspected bots can change their ip out of ranges and also they do not have to declare accurate user agent strings . So there is still acess to my online website by all the major back link bots.

Is there something i overlooked perhaps a php solution i tired crawling the competitor whom i suspected and i can get anything on him he managed to do it maybe with redirects im not sure how he identifies the bots and spiders any ideas?