Saving bandwidth by preventing spider access to your wordpress weblog.
出典: くみこみックス
I have a couple of internet sites, and just not too long ago (this past month), I've noticed wordpress spiderspanker that 1 of them is making use of what I would think about excessive bandwidth. Digging a little deeper, it appears that bots are making use of all the bandwidth.
This is just my private internet site – I have a travel weblog in the main domain and a household history site in a subdomain. In the past, I have employed on average 400mb per month of bandwidth. This month I've had to improve the bandwidth to 1.5GB, but it is possibly going to go over (it's at the moment at 1.38GB). These two sites aren't huge, and don't get stacks of hits, primarily just buddies and household. The loved ones history website has a couple of significant images, but absolutely nothing excessive.
Looking at awstats this month, the weblog website has utilized 150mb of bandwidth + 480mb of bot bandwidth (380mb of that is msnbot). The family members history website has used 55mb of bandwidth + 650mb of bot bandwidth (620mb of that is googlebot).
Most robots determine themselves by a custom user agent in the request headers. Which can easily be blocked with htaccess.
There are a quantity of great articles on this. Let me know if you have any problems, as it is a matter of identifying the offending bots/crawlers and banning them as per your want.
I have utilized google tools to tell it to not frequent the internet site as significantly about a week ago, but it doesn't seem to have created a distinct. Short of telling the robots to bugger off fully by means of the robots.txt file (it really is just my private site, but it is nonetheless nice to be listed in google!), is there something else I can do?
I at the moment use this on all my sites , essentially it blocks all bad user agents , poor bots and scrappers, Not only can it save your content from getting mass harvested but will also save you a little bandwidth simply because of less bots operating around your web site. Hope it helps
can tell you that Google drags in a great numerous spiders due to advertizing, specifically if you are making use of Adsense on your internet site along with ising the numerous ads from the Google ad network partners - these partners also send their bots to test your site visitors sources and what adverts to place on your website - Google has been hitting hard lately because of the algorithm tweaks and Adsense possessing had a profitable month in the terms of the quantity of new advertisers on board.
Even if you slow down the crawl rate, you will nonetheless see a huge chunk of bandwidth disappear. The bots are way too intermittent to make correct adjustments unless you wish to block them.
There's a new wordpress plugin that can assist with this! I've gotten a couple of emails re a item being sold to get rid of or "spank" the undesirable spiders that are taking up lots of bandwidth and not adding worth to your business, freeing up space for genuine guests and not causing a difficulty with hosting limits. It really is named Spyder spanker at that name .com if you want to see the sales page.
Anyway, I'm not sure if this is one thing helpful that I want or not. I do see a lot of spider wordpress [http://www.blogsense-wp.com/news/wordpress-prevent-spider-access/ spynderspanker review spider management] activity in my stats, but I usually believed that was sort of great b/c it means they are crawling my websites and hopefully indexing them.
The huge danger is stealing your bandwidth. Some of the spiders sent by spammers will hammer your web site as rapidly as they can, slowing down response for your human visitors.
